vector icon indicating copy to clipboard operation
vector copied to clipboard
verified publisher icon vectordotdev

feature(kafka source): autogen docs

Open davidhuie-dd opened this issue 3 years ago • 2 comments

Closes #15842

davidhuie-dd avatar Jan 12 '23 17:01 davidhuie-dd

Deploy Preview for vrl-playground canceled.

Name Link
Latest commit d1591255a03d1e5d752f6e60bc666386cce670c7
Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/63c1bd03c128de000847bf0e

netlify[bot] avatar Jan 12 '23 17:01 netlify[bot]

Deploy Preview for vector-project canceled.

Name Link
Latest commit d1591255a03d1e5d752f6e60bc666386cce670c7
Latest deploy log https://app.netlify.com/sites/vector-project/deploys/63c1bd03fa140d0007ba3bbe

netlify[bot] avatar Jan 12 '23 17:01 netlify[bot]

Regression Test Results

Run ID: 68054b8e-fa9a-4ba0-b385-eb17ded89a5e
Baseline: f0d3f1b8a5bf7090dbcf3a589badb4f7bc1433fd
Comparison: eb32521ed993bdd12611fef9ecb3d8e059952dcb
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
datadog_agent_remap_blackhole 1.09MiB/CPU-s 3.61 100.00% 30.18MiB/CPU-s 1.6MiB/CPU-s 21.17KiB/CPU-s 0.0 0.053051 31.28MiB/CPU-s 861.33KiB/CPU-s 11.12KiB/CPU-s 0.0 0.026893 False False
splunk_hec_route_s3 329.75KiB/CPU-s 2.82 100.00% 11.42MiB/CPU-s 682.05KiB/CPU-s 8.8KiB/CPU-s 0.0 0.058341 11.74MiB/CPU-s 560.81KiB/CPU-s 7.24KiB/CPU-s 0.0 0.046655 False False
socket_to_socket_blackhole 337.06KiB/CPU-s 2.51 100.00% 13.11MiB/CPU-s 334.93KiB/CPU-s 4.32KiB/CPU-s 0.0 0.024946 13.44MiB/CPU-s 123.29KiB/CPU-s 1.59KiB/CPU-s 0.0 0.008958 False False
syslog_loki 170.46KiB/CPU-s 1.92 100.00% 8.69MiB/CPU-s 339.05KiB/CPU-s 4.38KiB/CPU-s 0.0 0.038105 8.85MiB/CPU-s 236.1KiB/CPU-s 3.05KiB/CPU-s 0.0 0.026036 False False
syslog_log2metric_splunk_hec_metrics 168.28KiB/CPU-s 1.79 100.00% 9.19MiB/CPU-s 369.07KiB/CPU-s 4.76KiB/CPU-s 0.0 0.039209 9.36MiB/CPU-s 345.16KiB/CPU-s 4.46KiB/CPU-s 0.0 0.036024 False False
otlp_grpc_to_blackhole 18.16KiB/CPU-s 1.75 100.00% 1.01MiB/CPU-s 41.01KiB/CPU-s 542.1B/CPU-s 0.0 0.039486 1.03MiB/CPU-s 53.98KiB/CPU-s 713.27B/CPU-s 0.0 0.051086 False False
syslog_humio_logs 85.16KiB/CPU-s 0.93 100.00% 8.93MiB/CPU-s 597.39KiB/CPU-s 7.71KiB/CPU-s 0.0 0.06531 9.02MiB/CPU-s 341.28KiB/CPU-s 4.41KiB/CPU-s 0.0 0.036967 False False
otlp_http_to_blackhole 11.42KiB/CPU-s 0.75 100.00% 1.49MiB/CPU-s 122.79KiB/CPU-s 1.58KiB/CPU-s 0.0 0.080308 1.5MiB/CPU-s 134.64KiB/CPU-s 1.74KiB/CPU-s 0.0 0.087405 False False
syslog_splunk_hec_logs 42.93KiB/CPU-s 0.46 100.00% 9.02MiB/CPU-s 203.02KiB/CPU-s 2.62KiB/CPU-s 0.0 0.02197 9.07MiB/CPU-s 282.76KiB/CPU-s 3.65KiB/CPU-s 0.0 0.030457 False False
syslog_regex_logs2metric_ddmetrics 16.95KiB/CPU-s 0.45 94.68% 3.67MiB/CPU-s 486.33KiB/CPU-s 6.28KiB/CPU-s 0.0 0.129538 3.68MiB/CPU-s 473.97KiB/CPU-s 6.12KiB/CPU-s 0.0 0.125678 True False
http_to_http_noack 10.19KiB/CPU-s 0.07 92.07% 13.61MiB/CPU-s 368.3KiB/CPU-s 4.75KiB/CPU-s 0.0 0.026429 13.62MiB/CPU-s 258.33KiB/CPU-s 3.33KiB/CPU-s 0.0 0.018524 False False
splunk_hec_to_splunk_hec_logs_noack 9.17KiB/CPU-s 0.07 97.25% 13.62MiB/CPU-s 258.68KiB/CPU-s 3.34KiB/CPU-s 0.0 0.018551 13.63MiB/CPU-s 192.2KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013774 False False
http_text_to_http_json 13.55KiB/CPU-s 0.06 76.69% 24.04MiB/CPU-s 698.25KiB/CPU-s 9.01KiB/CPU-s 0.0 0.028367 24.05MiB/CPU-s 536.28KiB/CPU-s 6.92KiB/CPU-s 0.0 0.021775 False False
enterprise_http_to_http 8.34KiB/CPU-s 0.06 92.38% 13.61MiB/CPU-s 308.23KiB/CPU-s 3.98KiB/CPU-s 0.0 0.022108 13.62MiB/CPU-s 193.92KiB/CPU-s 2.5KiB/CPU-s 0.0 0.013901 False False
splunk_hec_to_splunk_hec_logs_acks 3.31KiB/CPU-s 0.02 40.47% 13.61MiB/CPU-s 343.65KiB/CPU-s 4.43KiB/CPU-s 0.0 0.024648 13.62MiB/CPU-s 340.31KiB/CPU-s 4.39KiB/CPU-s 0.0 0.024403 False False
fluent_elasticsearch -184.36B/CPU-s -0.0 25.92% 45.41MiB/CPU-s 29.99KiB/CPU-s 392.13B/CPU-s 0.0 0.000645 45.41MiB/CPU-s 30.31KiB/CPU-s 396.07B/CPU-s 0.0 0.000652 False False
splunk_hec_indexer_ack_blackhole -2.0KiB/CPU-s -0.01 33.74% 13.62MiB/CPU-s 246.42KiB/CPU-s 3.18KiB/CPU-s 0.0 0.01767 13.62MiB/CPU-s 254.79KiB/CPU-s 3.29KiB/CPU-s 0.0 0.018273 False False
file_to_blackhole -12.3KiB/CPU-s -0.02 43.49% 54.5MiB/CPU-s 1.12MiB/CPU-s 14.8KiB/CPU-s 0.0 0.020562 54.49MiB/CPU-s 1.17MiB/CPU-s 15.42KiB/CPU-s 0.0 0.021435 False False
http_to_http_json -15.83KiB/CPU-s -0.11 99.98% 13.62MiB/CPU-s 211.43KiB/CPU-s 2.73KiB/CPU-s 0.0 0.015156 13.61MiB/CPU-s 251.66KiB/CPU-s 3.25KiB/CPU-s 0.0 0.018061 False False
http_to_http_acks -14.3KiB/CPU-s -0.27 21.84% 5.22MiB/CPU-s 2.72MiB/CPU-s 35.94KiB/CPU-s 0.0 0.520586 5.21MiB/CPU-s 2.8MiB/CPU-s 36.97KiB/CPU-s 0.0 0.536882 True False
syslog_log2metric_humio_metrics -34.11KiB/CPU-s -0.55 100.00% 6.05MiB/CPU-s 347.91KiB/CPU-s 4.49KiB/CPU-s 0.0 0.056174 6.01MiB/CPU-s 328.07KiB/CPU-s 4.23KiB/CPU-s 0.0 0.053263 False False
datadog_agent_remap_datadog_logs_acks -492.21KiB/CPU-s -1.43 100.00% 33.61MiB/CPU-s 1.48MiB/CPU-s 19.49KiB/CPU-s 0.0 0.043889 33.13MiB/CPU-s 1.47MiB/CPU-s 19.38KiB/CPU-s 0.0 0.044264 False False
datadog_agent_remap_blackhole_acks -861.13KiB/CPU-s -2.69 100.00% 31.21MiB/CPU-s 1.31MiB/CPU-s 17.38KiB/CPU-s 0.0 0.042106 30.37MiB/CPU-s 1.59MiB/CPU-s 21.07KiB/CPU-s 0.0 0.052474 False False
datadog_agent_remap_datadog_logs -1.16MiB/CPU-s -3.37 100.00% 34.24MiB/CPU-s 1.18MiB/CPU-s 15.62KiB/CPU-s 0.0 0.03453 33.09MiB/CPU-s 1.69MiB/CPU-s 22.35KiB/CPU-s 0.0 0.051134 False False

github-actions[bot] avatar Jan 12 '23 18:01 github-actions[bot]

Regression Test Results

Run ID: 5e61980e-5b46-4fef-ab23-0001d5eb3daa
Baseline: f90ca9e3e82b7577608b0c82a5feffb3f96d5f4c
Comparison: 040ad06f864d094dd5b5f4d0895adab5ef73de06
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_splunk_hec_metrics 288.0KiB/CPU-s 3.03 100.00% 9.27MiB/CPU-s 430.82KiB/CPU-s 5.56KiB/CPU-s 0.0 0.045373 9.55MiB/CPU-s 132.18KiB/CPU-s 1.71KiB/CPU-s 0.0 0.013511 False False
syslog_log2metric_humio_metrics 66.84KiB/CPU-s 1.08 100.00% 6.07MiB/CPU-s 337.99KiB/CPU-s 4.36KiB/CPU-s 0.0 0.054412 6.13MiB/CPU-s 392.7KiB/CPU-s 5.07KiB/CPU-s 0.0 0.062546 False False
splunk_hec_route_s3 93.44KiB/CPU-s 0.79 100.00% 11.56MiB/CPU-s 661.74KiB/CPU-s 8.54KiB/CPU-s 0.0 0.055902 11.65MiB/CPU-s 627.86KiB/CPU-s 8.1KiB/CPU-s 0.0 0.052625 False False
datadog_agent_remap_blackhole 241.78KiB/CPU-s 0.77 100.00% 30.7MiB/CPU-s 2.05MiB/CPU-s 27.03KiB/CPU-s 0.0 0.066632 30.94MiB/CPU-s 2.32MiB/CPU-s 30.66KiB/CPU-s 0.0 0.074977 False False
syslog_regex_logs2metric_ddmetrics 28.12KiB/CPU-s 0.74 99.81% 3.69MiB/CPU-s 500.61KiB/CPU-s 6.46KiB/CPU-s 0.0 0.132534 3.72MiB/CPU-s 491.58KiB/CPU-s 6.35KiB/CPU-s 0.0 0.129181 True False
http_to_http_acks 36.34KiB/CPU-s 0.67 51.24% 5.26MiB/CPU-s 2.78MiB/CPU-s 36.78KiB/CPU-s 0.0 0.528912 5.3MiB/CPU-s 2.82MiB/CPU-s 37.26KiB/CPU-s 0.0 0.532219 True False
enterprise_http_to_http 10.57KiB/CPU-s 0.08 96.74% 13.61MiB/CPU-s 331.47KiB/CPU-s 4.28KiB/CPU-s 0.0 0.023779 13.62MiB/CPU-s 192.29KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013784 False False
http_to_http_noack 6.2KiB/CPU-s 0.04 70.91% 13.61MiB/CPU-s 354.23KiB/CPU-s 4.57KiB/CPU-s 0.0 0.025416 13.62MiB/CPU-s 285.56KiB/CPU-s 3.69KiB/CPU-s 0.0 0.020479 False False
splunk_hec_to_splunk_hec_logs_acks 3.28KiB/CPU-s 0.02 40.34% 13.62MiB/CPU-s 342.59KiB/CPU-s 4.42KiB/CPU-s 0.0 0.024567 13.62MiB/CPU-s 335.98KiB/CPU-s 4.34KiB/CPU-s 0.0 0.024087 False False
splunk_hec_to_splunk_hec_logs_noack 2.2KiB/CPU-s 0.02 44.96% 13.62MiB/CPU-s 210.3KiB/CPU-s 2.71KiB/CPU-s 0.0 0.015074 13.63MiB/CPU-s 192.2KiB/CPU-s 2.48KiB/CPU-s 0.0 0.013774 False False
fluent_elasticsearch 310.09B/CPU-s 0.0 42.26% 45.41MiB/CPU-s 29.93KiB/CPU-s 391.29B/CPU-s 0.0 0.000644 45.41MiB/CPU-s 30.29KiB/CPU-s 395.83B/CPU-s 0.0 0.000651 False False
file_to_blackhole -4.15KiB/CPU-s -0.01 15.71% 54.49MiB/CPU-s 1.1MiB/CPU-s 14.59KiB/CPU-s 0.0 0.020268 54.49MiB/CPU-s 1.14MiB/CPU-s 15.04KiB/CPU-s 0.0 0.020907 False False
splunk_hec_indexer_ack_blackhole -2.35KiB/CPU-s -0.02 40.53% 13.62MiB/CPU-s 237.05KiB/CPU-s 3.06KiB/CPU-s 0.0 0.016995 13.62MiB/CPU-s 246.45KiB/CPU-s 3.18KiB/CPU-s 0.0 0.017672 False False
datadog_agent_remap_datadog_logs_acks -38.61KiB/CPU-s -0.11 95.98% 34.02MiB/CPU-s 1.09MiB/CPU-s 14.42KiB/CPU-s 0.0 0.032085 33.98MiB/CPU-s 936.12KiB/CPU-s 12.08KiB/CPU-s 0.0 0.026903 False False
http_to_http_json -44.57KiB/CPU-s -0.32 100.00% 13.62MiB/CPU-s 227.97KiB/CPU-s 2.94KiB/CPU-s 0.0 0.016347 13.57MiB/CPU-s 283.48KiB/CPU-s 3.66KiB/CPU-s 0.0 0.020392 False False
datadog_agent_remap_datadog_logs -206.74KiB/CPU-s -0.59 100.00% 34.22MiB/CPU-s 1.18MiB/CPU-s 15.63KiB/CPU-s 0.0 0.034579 34.02MiB/CPU-s 1.27MiB/CPU-s 16.82KiB/CPU-s 0.0 0.037436 False False
socket_to_socket_blackhole -129.93KiB/CPU-s -0.95 100.00% 13.37MiB/CPU-s 232.82KiB/CPU-s 3.01KiB/CPU-s 0.0 0.017 13.25MiB/CPU-s 199.51KiB/CPU-s 2.58KiB/CPU-s 0.0 0.014708 False False
syslog_loki -101.82KiB/CPU-s -1.11 100.00% 8.93MiB/CPU-s 172.5KiB/CPU-s 2.23KiB/CPU-s 0.0 0.018856 8.83MiB/CPU-s 239.39KiB/CPU-s 3.09KiB/CPU-s 0.0 0.026463 False False
otlp_grpc_to_blackhole -14.73KiB/CPU-s -1.37 100.00% 1.05MiB/CPU-s 45.91KiB/CPU-s 606.73B/CPU-s 0.0 0.042664 1.04MiB/CPU-s 52.46KiB/CPU-s 693.18B/CPU-s 0.0 0.049423 False False
syslog_splunk_hec_logs -155.64KiB/CPU-s -1.65 100.00% 9.23MiB/CPU-s 72.54KiB/CPU-s 958.98B/CPU-s 0.0 0.007675 9.08MiB/CPU-s 187.62KiB/CPU-s 2.42KiB/CPU-s 0.0 0.020182 False False
http_text_to_http_json -431.02KiB/CPU-s -1.73 100.00% 24.33MiB/CPU-s 605.64KiB/CPU-s 7.82KiB/CPU-s 0.0 0.02431 23.91MiB/CPU-s 508.21KiB/CPU-s 6.56KiB/CPU-s 0.0 0.020759 False False
syslog_humio_logs -177.12KiB/CPU-s -1.87 100.00% 9.27MiB/CPU-s 163.31KiB/CPU-s 2.11KiB/CPU-s 0.0 0.01721 9.09MiB/CPU-s 271.25KiB/CPU-s 3.5KiB/CPU-s 0.0 0.029127 False False
datadog_agent_remap_blackhole_acks -607.98KiB/CPU-s -1.88 100.00% 31.54MiB/CPU-s 1.25MiB/CPU-s 16.5KiB/CPU-s 0.0 0.039552 30.95MiB/CPU-s 1.3MiB/CPU-s 17.25KiB/CPU-s 0.0 0.04215 False False
otlp_http_to_blackhole -32.17KiB/CPU-s -2.02 100.00% 1.55MiB/CPU-s 106.54KiB/CPU-s 1.37KiB/CPU-s 0.0 0.066992 1.52MiB/CPU-s 124.01KiB/CPU-s 1.6KiB/CPU-s 0.0 0.079589 False False

github-actions[bot] avatar Jan 12 '23 19:01 github-actions[bot]

Regression Test Results

Run ID: 8c6526ea-56db-4eb8-9d77-8421125aa383
Baseline: 10547f97c05164670f6ae588436d0031cd06f20f
Comparison: 3adc9d8171b27339ea20c041c510134cd0e47159
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
otlp_grpc_to_blackhole 75.19KiB/CPU-s 7.7 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
otlp_grpc_to_blackhole 75.19KiB/CPU-s 7.7 100.00% 975.97KiB/CPU-s 294.84KiB/CPU-s 3.81KiB/CPU-s 0.0 0.30207 1.03MiB/CPU-s 57.89KiB/CPU-s 764.91B/CPU-s 0.0 0.055072 True False
syslog_log2metric_humio_metrics 179.38KiB/CPU-s 3.01 100.00% 5.82MiB/CPU-s 379.43KiB/CPU-s 4.9KiB/CPU-s 0.0 0.063629 6.0MiB/CPU-s 363.17KiB/CPU-s 4.69KiB/CPU-s 0.0 0.059123 False False
syslog_log2metric_splunk_hec_metrics 171.24KiB/CPU-s 1.78 100.00% 9.42MiB/CPU-s 439.32KiB/CPU-s 5.67KiB/CPU-s 0.0 0.045564 9.58MiB/CPU-s 346.79KiB/CPU-s 4.48KiB/CPU-s 0.0 0.03534 False False
syslog_regex_logs2metric_ddmetrics 51.44KiB/CPU-s 1.44 100.00% 3.48MiB/CPU-s 417.89KiB/CPU-s 5.39KiB/CPU-s 0.0 0.117347 3.53MiB/CPU-s 409.77KiB/CPU-s 5.29KiB/CPU-s 0.0 0.113428 True False
syslog_loki 124.42KiB/CPU-s 1.41 100.00% 8.59MiB/CPU-s 522.02KiB/CPU-s 6.74KiB/CPU-s 0.0 0.059354 8.71MiB/CPU-s 290.31KiB/CPU-s 3.75KiB/CPU-s 0.0 0.032548 False False
datadog_agent_remap_datadog_logs_acks 391.39KiB/CPU-s 1.16 100.00% 33.06MiB/CPU-s 1.7MiB/CPU-s 22.41KiB/CPU-s 0.0 0.051313 33.44MiB/CPU-s 1.35MiB/CPU-s 17.88KiB/CPU-s 0.0 0.040453 False False
http_to_http_acks 43.32KiB/CPU-s 0.81 59.08% 5.25MiB/CPU-s 2.81MiB/CPU-s 37.13KiB/CPU-s 0.0 0.534616 5.3MiB/CPU-s 2.81MiB/CPU-s 37.1KiB/CPU-s 0.0 0.529924 True False
datadog_agent_remap_blackhole 185.96KiB/CPU-s 0.59 100.00% 30.77MiB/CPU-s 1.35MiB/CPU-s 17.86KiB/CPU-s 0.0 0.043907 30.95MiB/CPU-s 1.47MiB/CPU-s 19.39KiB/CPU-s 0.0 0.047398 False False
http_to_http_json 58.02KiB/CPU-s 0.42 100.00% 13.57MiB/CPU-s 303.53KiB/CPU-s 3.92KiB/CPU-s 0.0 0.021849 13.62MiB/CPU-s 210.21KiB/CPU-s 2.71KiB/CPU-s 0.0 0.015068 False False
enterprise_http_to_http 14.47KiB/CPU-s 0.1 98.64% 13.61MiB/CPU-s 392.85KiB/CPU-s 5.07KiB/CPU-s 0.0 0.028195 13.62MiB/CPU-s 228.51KiB/CPU-s 2.95KiB/CPU-s 0.0 0.016383 False False
splunk_hec_to_splunk_hec_logs_noack 6.52KiB/CPU-s 0.05 89.92% 13.62MiB/CPU-s 243.59KiB/CPU-s 3.14KiB/CPU-s 0.0 0.017466 13.62MiB/CPU-s 187.84KiB/CPU-s 2.42KiB/CPU-s 0.0 0.013462 False False
http_to_http_noack 2.14KiB/CPU-s 0.02 28.98% 13.61MiB/CPU-s 326.17KiB/CPU-s 4.21KiB/CPU-s 0.0 0.023399 13.61MiB/CPU-s 303.97KiB/CPU-s 3.92KiB/CPU-s 0.0 0.021803 False False
splunk_hec_to_splunk_hec_logs_acks 2.07KiB/CPU-s 0.01 25.70% 13.62MiB/CPU-s 353.54KiB/CPU-s 4.56KiB/CPU-s 0.0 0.025356 13.62MiB/CPU-s 336.8KiB/CPU-s 4.35KiB/CPU-s 0.0 0.024152 False False
fluent_elasticsearch -545.51B/CPU-s -0.0 67.34% 45.41MiB/CPU-s 29.98KiB/CPU-s 391.87B/CPU-s 0.0 0.000645 45.41MiB/CPU-s 30.18KiB/CPU-s 394.46B/CPU-s 0.0 0.000649 False False
splunk_hec_indexer_ack_blackhole -1.8KiB/CPU-s -0.01 30.51% 13.62MiB/CPU-s 246.44KiB/CPU-s 3.18KiB/CPU-s 0.0 0.017671 13.62MiB/CPU-s 257.03KiB/CPU-s 3.32KiB/CPU-s 0.0 0.018433 False False
file_to_blackhole -11.6KiB/CPU-s -0.02 40.61% 54.49MiB/CPU-s 1.04MiB/CPU-s 13.79KiB/CPU-s 0.0 0.019162 54.48MiB/CPU-s 1.27MiB/CPU-s 16.81KiB/CPU-s 0.0 0.023382 False False
datadog_agent_remap_blackhole_acks -39.25KiB/CPU-s -0.12 97.43% 31.89MiB/CPU-s 1.06MiB/CPU-s 14.04KiB/CPU-s 0.0 0.033293 31.85MiB/CPU-s 821.27KiB/CPU-s 10.61KiB/CPU-s 0.0 0.025177 False False
splunk_hec_route_s3 -46.23KiB/CPU-s -0.39 99.94% 11.44MiB/CPU-s 743.07KiB/CPU-s 9.59KiB/CPU-s 0.0 0.063415 11.4MiB/CPU-s 733.77KiB/CPU-s 9.47KiB/CPU-s 0.0 0.06287 False False
syslog_splunk_hec_logs -37.26KiB/CPU-s -0.4 100.00% 9.03MiB/CPU-s 190.61KiB/CPU-s 2.46KiB/CPU-s 0.0 0.02061 8.99MiB/CPU-s 304.77KiB/CPU-s 3.93KiB/CPU-s 0.0 0.033087 False False
otlp_http_to_blackhole -6.44KiB/CPU-s -0.41 99.58% 1.52MiB/CPU-s 118.04KiB/CPU-s 1.52KiB/CPU-s 0.0 0.075805 1.51MiB/CPU-s 128.52KiB/CPU-s 1.66KiB/CPU-s 0.0 0.082879 False False
datadog_agent_remap_datadog_logs -231.64KiB/CPU-s -0.68 100.00% 33.48MiB/CPU-s 1.91MiB/CPU-s 25.29KiB/CPU-s 0.0 0.057188 33.25MiB/CPU-s 1.62MiB/CPU-s 21.4KiB/CPU-s 0.0 0.048716 False False
http_text_to_http_json -348.71KiB/CPU-s -1.43 100.00% 23.77MiB/CPU-s 655.16KiB/CPU-s 8.46KiB/CPU-s 0.0 0.026913 23.43MiB/CPU-s 832.65KiB/CPU-s 10.75KiB/CPU-s 0.0 0.034701 False False
syslog_humio_logs -199.65KiB/CPU-s -2.11 100.00% 9.25MiB/CPU-s 180.52KiB/CPU-s 2.33KiB/CPU-s 0.0 0.019064 9.05MiB/CPU-s 396.24KiB/CPU-s 5.11KiB/CPU-s 0.0 0.042748 False False
socket_to_socket_blackhole -353.2KiB/CPU-s -2.62 100.00% 13.18MiB/CPU-s 326.92KiB/CPU-s 4.22KiB/CPU-s 0.0 0.024214 12.84MiB/CPU-s 413.77KiB/CPU-s 5.34KiB/CPU-s 0.0 0.03147 False False

github-actions[bot] avatar Jan 12 '23 23:01 github-actions[bot]

Regression Test Results

Run ID: 6e3a54a0-9e3b-40ec-bd5f-1ae85d19357b
Baseline: 81072b74bfe8e12ed8585d3770c2d440f6d1d68e
Comparison: 60f6f97412dcceac2e39e3df887e08f9268ff4a0
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%.

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_splunk_hec_metrics 244.84KiB/CPU-s 2.63 100.00% 9.08MiB/CPU-s 433.99KiB/CPU-s 5.6KiB/CPU-s 0.0 0.04669 9.32MiB/CPU-s 254.6KiB/CPU-s 3.29KiB/CPU-s 0.0 0.026688 False False
http_to_http_acks 116.92KiB/CPU-s 2.18 97.27% 5.23MiB/CPU-s 2.86MiB/CPU-s 37.81KiB/CPU-s 0.0 0.546813 5.35MiB/CPU-s 2.81MiB/CPU-s 37.08KiB/CPU-s 0.0 0.524786 True False
datadog_agent_remap_datadog_logs 163.78KiB/CPU-s 0.48 100.00% 33.31MiB/CPU-s 1.77MiB/CPU-s 23.35KiB/CPU-s 0.0 0.053061 33.47MiB/CPU-s 1.66MiB/CPU-s 21.94KiB/CPU-s 0.0 0.04962 False False
http_text_to_http_json 113.85KiB/CPU-s 0.47 100.00% 23.88MiB/CPU-s 993.53KiB/CPU-s 12.82KiB/CPU-s 0.0 0.040627 23.99MiB/CPU-s 872.89KiB/CPU-s 11.26KiB/CPU-s 0.0 0.035528 False False
datadog_agent_remap_blackhole_acks 100.83KiB/CPU-s 0.32 100.00% 31.21MiB/CPU-s 705.6KiB/CPU-s 9.11KiB/CPU-s 0.0 0.022079 31.3MiB/CPU-s 810.44KiB/CPU-s 10.46KiB/CPU-s 0.0 0.02528 False False
syslog_humio_logs 10.94KiB/CPU-s 0.12 99.70% 8.93MiB/CPU-s 230.7KiB/CPU-s 2.98KiB/CPU-s 0.0 0.025224 8.94MiB/CPU-s 168.27KiB/CPU-s 2.17KiB/CPU-s 0.0 0.018376 False False
enterprise_http_to_http 6.37KiB/CPU-s 0.05 75.96% 13.61MiB/CPU-s 331.31KiB/CPU-s 4.27KiB/CPU-s 0.0 0.023767 13.62MiB/CPU-s 258.63KiB/CPU-s 3.34KiB/CPU-s 0.0 0.018545 False False
splunk_hec_to_splunk_hec_logs_noack 3.31KiB/CPU-s 0.02 61.99% 13.62MiB/CPU-s 220.22KiB/CPU-s 2.84KiB/CPU-s 0.0 0.015786 13.63MiB/CPU-s 191.62KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013733 False False
splunk_hec_to_splunk_hec_logs_acks 2.92KiB/CPU-s 0.02 34.19% 13.61MiB/CPU-s 363.7KiB/CPU-s 4.69KiB/CPU-s 0.0 0.026086 13.62MiB/CPU-s 360.72KiB/CPU-s 4.65KiB/CPU-s 0.0 0.025867 False False
file_to_blackhole 4.47KiB/CPU-s 0.01 17.16% 54.49MiB/CPU-s 1.12MiB/CPU-s 14.8KiB/CPU-s 0.0 0.020556 54.5MiB/CPU-s 1.09MiB/CPU-s 14.35KiB/CPU-s 0.0 0.019942 False False
fluent_elasticsearch -46.06B/CPU-s -0.0 6.62% 45.41MiB/CPU-s 30.1KiB/CPU-s 393.41B/CPU-s 0.0 0.000647 45.41MiB/CPU-s 29.94KiB/CPU-s 391.32B/CPU-s 0.0 0.000644 False False
http_to_http_noack -104.03B/CPU-s -0.0 1.35% 13.61MiB/CPU-s 330.63KiB/CPU-s 4.27KiB/CPU-s 0.0 0.023719 13.61MiB/CPU-s 327.97KiB/CPU-s 4.23KiB/CPU-s 0.0 0.023528 False False
splunk_hec_indexer_ack_blackhole -441.04B/CPU-s -0.0 7.41% 13.62MiB/CPU-s 252.54KiB/CPU-s 3.26KiB/CPU-s 0.0 0.01811 13.62MiB/CPU-s 255.02KiB/CPU-s 3.29KiB/CPU-s 0.0 0.018288 False False
http_to_http_json -26.86KiB/CPU-s -0.19 100.00% 13.59MiB/CPU-s 264.1KiB/CPU-s 3.41KiB/CPU-s 0.0 0.01898 13.56MiB/CPU-s 291.79KiB/CPU-s 3.76KiB/CPU-s 0.0 0.02101 False False
otlp_grpc_to_blackhole -4.69KiB/CPU-s -0.44 100.00% 1.04MiB/CPU-s 47.56KiB/CPU-s 628.51B/CPU-s 0.0 0.044599 1.04MiB/CPU-s 50.19KiB/CPU-s 663.19B/CPU-s 0.0 0.047271 False False
datadog_agent_remap_blackhole -177.7KiB/CPU-s -0.56 100.00% 30.89MiB/CPU-s 1.48MiB/CPU-s 19.59KiB/CPU-s 0.0 0.047979 30.72MiB/CPU-s 1.43MiB/CPU-s 18.95KiB/CPU-s 0.0 0.04668 False False
datadog_agent_remap_datadog_logs_acks -229.68KiB/CPU-s -0.67 100.00% 33.52MiB/CPU-s 1.29MiB/CPU-s 17.08KiB/CPU-s 0.0 0.03857 33.3MiB/CPU-s 1.03MiB/CPU-s 13.65KiB/CPU-s 0.0 0.031017 False False
splunk_hec_route_s3 -120.68KiB/CPU-s -1.0 100.00% 11.79MiB/CPU-s 540.21KiB/CPU-s 6.97KiB/CPU-s 0.0 0.044751 11.67MiB/CPU-s 533.15KiB/CPU-s 6.88KiB/CPU-s 0.0 0.044612 False False
otlp_http_to_blackhole -18.52KiB/CPU-s -1.16 100.00% 1.56MiB/CPU-s 97.11KiB/CPU-s 1.25KiB/CPU-s 0.0 0.060806 1.54MiB/CPU-s 120.98KiB/CPU-s 1.56KiB/CPU-s 0.0 0.076645 False False
syslog_log2metric_humio_metrics -80.7KiB/CPU-s -1.33 100.00% 5.92MiB/CPU-s 243.81KiB/CPU-s 3.15KiB/CPU-s 0.0 0.040248 5.84MiB/CPU-s 251.71KiB/CPU-s 3.25KiB/CPU-s 0.0 0.042114 False False
syslog_loki -121.03KiB/CPU-s -1.38 100.00% 8.59MiB/CPU-s 333.58KiB/CPU-s 4.31KiB/CPU-s 0.0 0.037928 8.47MiB/CPU-s 336.57KiB/CPU-s 4.35KiB/CPU-s 0.0 0.038802 False False
socket_to_socket_blackhole -308.82KiB/CPU-s -2.25 100.00% 13.41MiB/CPU-s 323.91KiB/CPU-s 4.18KiB/CPU-s 0.0 0.023592 13.11MiB/CPU-s 271.83KiB/CPU-s 3.51KiB/CPU-s 0.0 0.020255 False False
syslog_regex_logs2metric_ddmetrics -85.62KiB/CPU-s -2.25 100.00% 3.71MiB/CPU-s 433.82KiB/CPU-s 5.6KiB/CPU-s 0.0 0.114064 3.63MiB/CPU-s 468.46KiB/CPU-s 6.05KiB/CPU-s 0.0 0.126008 True False
syslog_splunk_hec_logs -223.53KiB/CPU-s -2.46 100.00% 8.87MiB/CPU-s 343.14KiB/CPU-s 4.43KiB/CPU-s 0.0 0.037778 8.65MiB/CPU-s 391.53KiB/CPU-s 5.05KiB/CPU-s 0.0 0.044194 False False

github-actions[bot] avatar Jan 13 '23 18:01 github-actions[bot]

Thinking about this, it's probably worth a new issue to track (not rendering the librdkafka options example)

spencergilbert avatar Jan 13 '23 19:01 spencergilbert

Regression Test Results

Run ID: 03f45273-b501-46ba-8dc9-0cd8d033b81e
Baseline: 6d05e13337bf8daacf47f0e289a521408fe079cc
Comparison: d1591255a03d1e5d752f6e60bc666386cce670c7
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparison SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

Changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

experiment Δ mean Δ mean % confidence
datadog_agent_remap_blackhole 1.6MiB/CPU-s 5.44 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
datadog_agent_remap_blackhole 1.6MiB/CPU-s 5.44 100.00% 29.45MiB/CPU-s 1.45MiB/CPU-s 19.23KiB/CPU-s 0.0 0.049349 31.06MiB/CPU-s 1.08MiB/CPU-s 14.22KiB/CPU-s 0.0 0.03465 False False
datadog_agent_remap_blackhole_acks 1.36MiB/CPU-s 4.56 100.00% 29.94MiB/CPU-s 1.91MiB/CPU-s 25.23KiB/CPU-s 0.0 0.0637 31.31MiB/CPU-s 1.4MiB/CPU-s 18.52KiB/CPU-s 0.0 0.044738 False False
syslog_splunk_hec_logs 270.49KiB/CPU-s 3.03 100.00% 8.73MiB/CPU-s 320.73KiB/CPU-s 4.14KiB/CPU-s 0.0 0.035866 9.0MiB/CPU-s 227.1KiB/CPU-s 2.93KiB/CPU-s 0.0 0.02465 False False
syslog_log2metric_humio_metrics 110.37KiB/CPU-s 1.8 100.00% 5.98MiB/CPU-s 388.89KiB/CPU-s 5.02KiB/CPU-s 0.0 0.063456 6.09MiB/CPU-s 323.53KiB/CPU-s 4.18KiB/CPU-s 0.0 0.051857 False False
http_to_http_acks 87.31KiB/CPU-s 1.63 90.68% 5.25MiB/CPU-s 2.79MiB/CPU-s 36.89KiB/CPU-s 0.0 0.532167 5.33MiB/CPU-s 2.77MiB/CPU-s 36.65KiB/CPU-s 0.0 0.520189 True False
datadog_agent_remap_datadog_logs 511.79KiB/CPU-s 1.48 100.00% 33.73MiB/CPU-s 1.28MiB/CPU-s 16.9KiB/CPU-s 0.0 0.037914 34.23MiB/CPU-s 1.18MiB/CPU-s 15.54KiB/CPU-s 0.0 0.03436 False False
splunk_hec_route_s3 114.64KiB/CPU-s 0.98 100.00% 11.48MiB/CPU-s 782.82KiB/CPU-s 10.1KiB/CPU-s 0.0 0.066579 11.59MiB/CPU-s 696.02KiB/CPU-s 8.98KiB/CPU-s 0.0 0.058625 False False
otlp_http_to_blackhole 14.94KiB/CPU-s 0.96 100.00% 1.52MiB/CPU-s 114.94KiB/CPU-s 1.48KiB/CPU-s 0.0 0.073805 1.54MiB/CPU-s 119.38KiB/CPU-s 1.54KiB/CPU-s 0.0 0.07593 False False
datadog_agent_remap_datadog_logs_acks 293.14KiB/CPU-s 0.85 100.00% 33.64MiB/CPU-s 1.39MiB/CPU-s 18.43KiB/CPU-s 0.0 0.041466 33.92MiB/CPU-s 902.38KiB/CPU-s 11.65KiB/CPU-s 0.0 0.025974 False False
syslog_loki 58.34KiB/CPU-s 0.66 100.00% 8.58MiB/CPU-s 337.46KiB/CPU-s 4.36KiB/CPU-s 0.0 0.038392 8.64MiB/CPU-s 282.68KiB/CPU-s 3.65KiB/CPU-s 0.0 0.031948 False False
http_to_http_json 30.42KiB/CPU-s 0.22 100.00% 13.59MiB/CPU-s 271.11KiB/CPU-s 3.5KiB/CPU-s 0.0 0.019476 13.62MiB/CPU-s 208.98KiB/CPU-s 2.7KiB/CPU-s 0.0 0.01498 False False
splunk_hec_to_splunk_hec_logs_noack 4.94KiB/CPU-s 0.04 79.90% 13.62MiB/CPU-s 230.28KiB/CPU-s 2.97KiB/CPU-s 0.0 0.016509 13.63MiB/CPU-s 191.4KiB/CPU-s 2.47KiB/CPU-s 0.0 0.013717 False False
socket_to_socket_blackhole 4.11KiB/CPU-s 0.03 56.89% 13.38MiB/CPU-s 308.02KiB/CPU-s 3.98KiB/CPU-s 0.0 0.022475 13.39MiB/CPU-s 262.49KiB/CPU-s 3.39KiB/CPU-s 0.0 0.019147 False False
http_to_http_noack 1.98KiB/CPU-s 0.01 31.14% 13.62MiB/CPU-s 284.15KiB/CPU-s 3.67KiB/CPU-s 0.0 0.020378 13.62MiB/CPU-s 256.48KiB/CPU-s 3.31KiB/CPU-s 0.0 0.018391 False False
enterprise_http_to_http 2.06KiB/CPU-s 0.01 35.69% 13.62MiB/CPU-s 258.71KiB/CPU-s 3.34KiB/CPU-s 0.0 0.018551 13.62MiB/CPU-s 228.42KiB/CPU-s 2.95KiB/CPU-s 0.0 0.016376 False False
fluent_elasticsearch -195.55B/CPU-s -0.0 27.61% 45.41MiB/CPU-s 29.99KiB/CPU-s 391.93B/CPU-s 0.0 0.000645 45.41MiB/CPU-s 29.89KiB/CPU-s 390.89B/CPU-s 0.0 0.000643 False False
splunk_hec_to_splunk_hec_logs_acks -612.09B/CPU-s -0.0 7.57% 13.62MiB/CPU-s 341.28KiB/CPU-s 4.4KiB/CPU-s 0.0 0.024474 13.62MiB/CPU-s 347.95KiB/CPU-s 4.49KiB/CPU-s 0.0 0.024954 False False
splunk_hec_indexer_ack_blackhole -1.32KiB/CPU-s -0.01 22.42% 13.62MiB/CPU-s 251.77KiB/CPU-s 3.25KiB/CPU-s 0.0 0.018054 13.62MiB/CPU-s 256.35KiB/CPU-s 3.31KiB/CPU-s 0.0 0.018384 False False
file_to_blackhole -9.3KiB/CPU-s -0.02 33.22% 54.5MiB/CPU-s 1.14MiB/CPU-s 15.1KiB/CPU-s 0.0 0.02098 54.49MiB/CPU-s 1.18MiB/CPU-s 15.55KiB/CPU-s 0.0 0.021615 False False
otlp_grpc_to_blackhole -3.56KiB/CPU-s -0.33 100.00% 1.04MiB/CPU-s 41.05KiB/CPU-s 542.56B/CPU-s 0.0 0.038355 1.04MiB/CPU-s 52.7KiB/CPU-s 696.28B/CPU-s 0.0 0.04941 False False
syslog_log2metric_splunk_hec_metrics -32.41KiB/CPU-s -0.34 100.00% 9.3MiB/CPU-s 352.04KiB/CPU-s 4.54KiB/CPU-s 0.0 0.036949 9.27MiB/CPU-s 394.87KiB/CPU-s 5.1KiB/CPU-s 0.0 0.041586 False False
syslog_humio_logs -69.1KiB/CPU-s -0.74 100.00% 9.08MiB/CPU-s 238.96KiB/CPU-s 3.08KiB/CPU-s 0.0 0.025704 9.01MiB/CPU-s 214.77KiB/CPU-s 2.77KiB/CPU-s 0.0 0.023275 False False
http_text_to_http_json -387.31KiB/CPU-s -1.58 100.00% 23.96MiB/CPU-s 933.55KiB/CPU-s 12.05KiB/CPU-s 0.0 0.03805 23.58MiB/CPU-s 925.65KiB/CPU-s 11.94KiB/CPU-s 0.0 0.038333 False False
syslog_regex_logs2metric_ddmetrics -69.86KiB/CPU-s -1.89 100.00% 3.61MiB/CPU-s 403.43KiB/CPU-s 5.21KiB/CPU-s 0.0 0.109226 3.54MiB/CPU-s 391.2KiB/CPU-s 5.05KiB/CPU-s 0.0 0.107958 True False

github-actions[bot] avatar Jan 13 '23 21:01 github-actions[bot]

@spencergilbert #15957 tracks the issues with HashMap examples.

davidhuie-dd avatar Jan 13 '23 21:01 davidhuie-dd