Motivation

Test with HEAD of main from system tests is inconsistent.

What Does This Do

Setup automation to upgrade the version with commit SHA periodically. It would create an automated pull requests to be review and merged.

The script is validated with

$ export TARGET=".github/workflows/run-system-tests.yaml"
$ export PATTERN='(\s*system-tests.yml@)(\S+)(\s+# Automated:.*)'
$ export REF=007f43b50f720be299f0dae27f4fcbabdd623591
$ .github/scripts/update_reference.sh

Target: .github/workflows/run-system-tests.yaml
Ref: 007f43b50f720be299f0dae27f4fcbabdd623591
Pattern: (\s*system-tests.yml@)(\S+)(\s+# Automated:.*)
✓ Updated references in .github/workflows/run-system-tests.yaml

$ git diff
diff --git a/.github/workflows/run-system-tests.yaml b/.github/workflows/run-system-tests.yaml
index bb320a46dd..feceb83dbb 100644
--- a/.github/workflows/run-system-tests.yaml
+++ b/.github/workflows/run-system-tests.yaml
@@ -50,7 +50,7 @@ jobs:
   main:
     needs:
     - build
-    uses: DataDog/system-tests/.github/workflows/system-tests.yml@main # Automated: This reference is automatically updated.
+    uses: DataDog/system-tests/.github/workflows/system-tests.yml@007f43b50f720be299f0dae27f4fcbabdd623591 # Automated: This reference is automatically updated.
     secrets: inherit
     permissions:
       contents: read

Additional Notes

The workflow requires to allow peter-evans/create-pull-request@* in the repository settings.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

Jun 04 '25 12:06 TonyCTHsu

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	tonycthsu/pin-system-tests
git_commit_date	1758657961	1758663646
git_commit_sha	11e677e239	f7017caa3c
release_version	1.54.0-SNAPSHOT~11e677e239	1.50.0-SNAPSHOT~f7017caa3c

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1758665527	1758665527
ci_job_id	1144344923	1144344923
ci_pipeline_id	77349259	77349259
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-df96pby3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-df96pby3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 6 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~f7017caa3c, baseline=1.54.0-SNAPSHOT~11e677e239

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.006 s) : 0, 1005647
Total [baseline] (8.654 s) : 0, 8654235
Agent [candidate] (1.009 s) : 0, 1008517
Total [candidate] (8.64 s) : 0, 8640001
section iast
Agent [baseline] (1.154 s) : 0, 1153544
Total [baseline] (9.342 s) : 0, 9342183
Agent [candidate] (1.146 s) : 0, 1145828
Total [candidate] (9.248 s) : 0, 9248206

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.006 s	-
Agent	iast	1.154 s	147.896 ms (14.7%)
Total	tracing	8.654 s	-
Total	iast	9.342 s	687.948 ms (7.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.009 s	-
Agent	iast	1.146 s	137.311 ms (13.6%)
Total	tracing	8.64 s	-
Total	iast	9.248 s	608.205 ms (7.0%)

gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~f7017caa3c, baseline=1.54.0-SNAPSHOT~11e677e239

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.467 ms) : 0, 1467
crashtracking [candidate] (1.453 ms) : 0, 1453
BytebuddyAgent [baseline] (688.427 ms) : 0, 688427
BytebuddyAgent [candidate] (690.392 ms) : 0, 690392
GlobalTracer [baseline] (247.503 ms) : 0, 247503
GlobalTracer [candidate] (248.179 ms) : 0, 248179
AppSec [baseline] (31.187 ms) : 0, 31187
AppSec [candidate] (31.303 ms) : 0, 31303
Debugger [baseline] (6.369 ms) : 0, 6369
Debugger [candidate] (6.389 ms) : 0, 6389
Remote Config [baseline] (676.057 µs) : 0, 676
Remote Config [candidate] (670.814 µs) : 0, 671
Telemetry [baseline] (9.014 ms) : 0, 9014
Telemetry [candidate] (9.065 ms) : 0, 9065
section iast
crashtracking [baseline] (1.464 ms) : 0, 1464
crashtracking [candidate] (1.445 ms) : 0, 1445
BytebuddyAgent [baseline] (815.995 ms) : 0, 815995
BytebuddyAgent [candidate] (811.481 ms) : 0, 811481
GlobalTracer [baseline] (239.36 ms) : 0, 239360
GlobalTracer [candidate] (237.042 ms) : 0, 237042
IAST [baseline] (26.419 ms) : 0, 26419
IAST [candidate] (25.942 ms) : 0, 25942
AppSec [baseline] (33.922 ms) : 0, 33922
AppSec [candidate] (33.68 ms) : 0, 33680
Debugger [baseline] (6.093 ms) : 0, 6093
Debugger [candidate] (6.054 ms) : 0, 6054
Remote Config [baseline] (589.762 µs) : 0, 590
Remote Config [candidate] (591.329 µs) : 0, 591
Telemetry [baseline] (8.376 ms) : 0, 8376
Telemetry [candidate] (8.338 ms) : 0, 8338

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~f7017caa3c, baseline=1.54.0-SNAPSHOT~11e677e239

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.014 s) : 0, 1014306
Total [baseline] (10.691 s) : 0, 10691095
Agent [candidate] (1.011 s) : 0, 1011005
Total [candidate] (10.743 s) : 0, 10742817
section appsec
Agent [baseline] (1.195 s) : 0, 1194650
Total [baseline] (11.034 s) : 0, 11034294
Agent [candidate] (1.185 s) : 0, 1184558
Total [candidate] (11.059 s) : 0, 11058829
section iast
Agent [baseline] (1.153 s) : 0, 1153434
Total [baseline] (10.965 s) : 0, 10964529
Agent [candidate] (1.153 s) : 0, 1152728
Total [candidate] (11.091 s) : 0, 11090850
section profiling
Agent [baseline] (1.154 s) : 0, 1153934
Total [baseline] (11.008 s) : 0, 11007704
Agent [candidate] (1.17 s) : 0, 1169662
Total [candidate] (11.117 s) : 0, 11117387

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.014 s	-
Agent	appsec	1.195 s	180.344 ms (17.8%)
Agent	iast	1.153 s	139.128 ms (13.7%)
Agent	profiling	1.154 s	139.628 ms (13.8%)
Total	tracing	10.691 s	-
Total	appsec	11.034 s	343.199 ms (3.2%)
Total	iast	10.965 s	273.434 ms (2.6%)
Total	profiling	11.008 s	316.609 ms (3.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.011 s	-
Agent	appsec	1.185 s	173.553 ms (17.2%)
Agent	iast	1.153 s	141.724 ms (14.0%)
Agent	profiling	1.17 s	158.658 ms (15.7%)
Total	tracing	10.743 s	-
Total	appsec	11.059 s	316.011 ms (2.9%)
Total	iast	11.091 s	348.032 ms (3.2%)
Total	profiling	11.117 s	374.57 ms (3.5%)

gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~f7017caa3c, baseline=1.54.0-SNAPSHOT~11e677e239

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.469 ms) : 0, 1469
crashtracking [candidate] (1.47 ms) : 0, 1470
BytebuddyAgent [baseline] (694.658 ms) : 0, 694658
BytebuddyAgent [candidate] (690.926 ms) : 0, 690926
GlobalTracer [baseline] (249.582 ms) : 0, 249582
GlobalTracer [candidate] (249.796 ms) : 0, 249796
AppSec [baseline] (31.258 ms) : 0, 31258
AppSec [candidate] (31.357 ms) : 0, 31357
Debugger [baseline] (6.417 ms) : 0, 6417
Debugger [candidate] (6.418 ms) : 0, 6418
Remote Config [baseline] (690.056 µs) : 0, 690
Remote Config [candidate] (679.359 µs) : 0, 679
Telemetry [baseline] (9.112 ms) : 0, 9112
Telemetry [candidate] (9.185 ms) : 0, 9185
section appsec
crashtracking [baseline] (1.456 ms) : 0, 1456
crashtracking [candidate] (1.442 ms) : 0, 1442
BytebuddyAgent [baseline] (717.397 ms) : 0, 717397
BytebuddyAgent [candidate] (710.837 ms) : 0, 710837
GlobalTracer [baseline] (241.016 ms) : 0, 241016
GlobalTracer [candidate] (239.349 ms) : 0, 239349
IAST [baseline] (24.663 ms) : 0, 24663
IAST [candidate] (24.572 ms) : 0, 24572
AppSec [baseline] (170.579 ms) : 0, 170579
AppSec [candidate] (172.189 ms) : 0, 172189
Debugger [baseline] (6.092 ms) : 0, 6092
Debugger [candidate] (5.972 ms) : 0, 5972
Remote Config [baseline] (649.441 µs) : 0, 649
Remote Config [candidate] (638.729 µs) : 0, 639
Telemetry [baseline] (11.599 ms) : 0, 11599
Telemetry [candidate] (8.466 ms) : 0, 8466
section iast
crashtracking [baseline] (1.458 ms) : 0, 1458
crashtracking [candidate] (1.46 ms) : 0, 1460
BytebuddyAgent [baseline] (816.775 ms) : 0, 816775
BytebuddyAgent [candidate] (815.807 ms) : 0, 815807
GlobalTracer [baseline] (238.5 ms) : 0, 238500
GlobalTracer [candidate] (238.469 ms) : 0, 238469
IAST [baseline] (26.452 ms) : 0, 26452
IAST [candidate] (26.231 ms) : 0, 26231
AppSec [baseline] (33.799 ms) : 0, 33799
AppSec [candidate] (33.223 ms) : 0, 33223
Debugger [baseline] (6.031 ms) : 0, 6031
Debugger [candidate] (7.051 ms) : 0, 7051
Remote Config [baseline] (632.621 µs) : 0, 633
Remote Config [candidate] (606.471 µs) : 0, 606
Telemetry [baseline] (8.374 ms) : 0, 8374
Telemetry [candidate] (8.476 ms) : 0, 8476
section profiling
crashtracking [baseline] (1.46 ms) : 0, 1460
crashtracking [candidate] (1.472 ms) : 0, 1472
BytebuddyAgent [baseline] (719.6 ms) : 0, 719600
BytebuddyAgent [candidate] (730.436 ms) : 0, 730436
GlobalTracer [baseline] (224.338 ms) : 0, 224338
GlobalTracer [candidate] (227.114 ms) : 0, 227114
AppSec [baseline] (31.373 ms) : 0, 31373
AppSec [candidate] (31.769 ms) : 0, 31769
Debugger [baseline] (6.518 ms) : 0, 6518
Debugger [candidate] (8.14 ms) : 0, 8140
Remote Config [baseline] (723.188 µs) : 0, 723
Remote Config [candidate] (741.815 µs) : 0, 742
Telemetry [baseline] (16.255 ms) : 0, 16255
Telemetry [candidate] (13.979 ms) : 0, 13979
ProfilingAgent [baseline] (101.592 ms) : 0, 101592
ProfilingAgent [candidate] (103.149 ms) : 0, 103149
Profiling [baseline] (102.195 ms) : 0, 102195
Profiling [candidate] (103.77 ms) : 0, 103770

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	tonycthsu/pin-system-tests
git_commit_date	1758657961	1758663646
git_commit_sha	11e677e239	f7017caa3c
release_version	1.54.0-SNAPSHOT~11e677e239	1.50.0-SNAPSHOT~f7017caa3c

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1758665193	1758665193
ci_job_id	1144344925	1144344925
ci_pipeline_id	77349259	77349259
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-orr3kg4q 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-orr3kg4q 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 3 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:profiling:high_load	worse [+205.257µs; +508.468µs] or [+2.370%; +5.870%]	unstable [-86.182op/s; +44.744op/s] or [-16.102%; +8.360%]	9.019ms	514.500op/s	8.662ms	535.219op/s
scenario:load:insecure-bank:iast_GLOBAL:high_load	worse [+337.986µs; +745.384µs] or [+3.232%; +7.127%]	unstable [-71.635op/s; +28.510op/s] or [-16.124%; +6.417%]	11.000ms	422.719op/s	10.459ms	444.281op/s
scenario:load:petclinic:profiling:high_load	worse [+1.434ms; +2.499ms] or [+2.978%; +5.190%]	unstable [-9.399op/s; +4.316op/s] or [-9.668%; +4.439%]	50.122ms	94.671op/s	48.155ms	97.213op/s
scenario:load:petclinic:appsec:high_load	better [-2.002ms; -1.079ms] or [-4.091%; -2.205%]	unstable [-3.962op/s; +10.162op/s] or [-4.145%; +10.630%]	47.401ms	98.700op/s	48.941ms	95.600op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~f7017caa3c, baseline=1.54.0-SNAPSHOT~11e677e239
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.467 ms) : 37167, 37766
.   : milestone, 37467,
appsec (48.941 ms) : 48504, 49379
.   : milestone, 48941,
code_origins (45.251 ms) : 44865, 45636
.   : milestone, 45251,
iast (45.056 ms) : 44678, 45434
.   : milestone, 45056,
profiling (48.155 ms) : 47708, 48603
.   : milestone, 48155,
tracing (44.396 ms) : 44016, 44776
.   : milestone, 44396,
section candidate
no_agent (36.802 ms) : 36501, 37103
.   : milestone, 36802,
appsec (47.401 ms) : 46981, 47820
.   : milestone, 47401,
code_origins (44.351 ms) : 43979, 44724
.   : milestone, 44351,
iast (46.274 ms) : 45884, 46664
.   : milestone, 46274,
profiling (50.122 ms) : 49583, 50660
.   : milestone, 50122,
tracing (44.022 ms) : 43635, 44409
.   : milestone, 44022,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.467 ms [37.167 ms, 37.766 ms]	-
appsec	48.941 ms [48.504 ms, 49.379 ms]	11.475 ms (30.6%)
code_origins	45.251 ms [44.865 ms, 45.636 ms]	7.784 ms (20.8%)
iast	45.056 ms [44.678 ms, 45.434 ms]	7.589 ms (20.3%)
profiling	48.155 ms [47.708 ms, 48.603 ms]	10.688 ms (28.5%)
tracing	44.396 ms [44.016 ms, 44.776 ms]	6.929 ms (18.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.802 ms [36.501 ms, 37.103 ms]	-
appsec	47.401 ms [46.981 ms, 47.82 ms]	10.598 ms (28.8%)
code_origins	44.351 ms [43.979 ms, 44.724 ms]	7.549 ms (20.5%)
iast	46.274 ms [45.884 ms, 46.664 ms]	9.472 ms (25.7%)
profiling	50.122 ms [49.583 ms, 50.66 ms]	13.32 ms (36.2%)
tracing	44.022 ms [43.635 ms, 44.409 ms]	7.22 ms (19.6%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~f7017caa3c, baseline=1.54.0-SNAPSHOT~11e677e239
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.438 ms) : 4388, 4489
.   : milestone, 4438,
iast (10.399 ms) : 10212, 10587
.   : milestone, 10399,
iast_FULL (14.065 ms) : 13782, 14348
.   : milestone, 14065,
iast_GLOBAL (10.459 ms) : 10274, 10643
.   : milestone, 10459,
profiling (8.662 ms) : 8528, 8796
.   : milestone, 8662,
tracing (7.724 ms) : 7607, 7841
.   : milestone, 7724,
section candidate
no_agent (4.352 ms) : 4294, 4409
.   : milestone, 4352,
iast (10.227 ms) : 10051, 10403
.   : milestone, 10227,
iast_FULL (14.223 ms) : 13945, 14500
.   : milestone, 14223,
iast_GLOBAL (11.0 ms) : 10806, 11194
.   : milestone, 11000,
profiling (9.019 ms) : 8871, 9167
.   : milestone, 9019,
tracing (7.967 ms) : 7854, 8081
.   : milestone, 7967,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.438 ms [4.388 ms, 4.489 ms]	-
iast	10.399 ms [10.212 ms, 10.587 ms]	5.961 ms (134.3%)
iast_FULL	14.065 ms [13.782 ms, 14.348 ms]	9.626 ms (216.9%)
iast_GLOBAL	10.459 ms [10.274 ms, 10.643 ms]	6.02 ms (135.6%)
profiling	8.662 ms [8.528 ms, 8.796 ms]	4.224 ms (95.2%)
tracing	7.724 ms [7.607 ms, 7.841 ms]	3.286 ms (74.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.352 ms [4.294 ms, 4.409 ms]	-
iast	10.227 ms [10.051 ms, 10.403 ms]	5.875 ms (135.0%)
iast_FULL	14.223 ms [13.945 ms, 14.5 ms]	9.871 ms (226.8%)
iast_GLOBAL	11.0 ms [10.806 ms, 11.194 ms]	6.648 ms (152.8%)
profiling	9.019 ms [8.871 ms, 9.167 ms]	4.667 ms (107.2%)
tracing	7.967 ms [7.854 ms, 8.081 ms]	3.615 ms (83.1%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	tonycthsu/pin-system-tests
git_commit_date	1758657961	1758663646
git_commit_sha	11e677e239	f7017caa3c
release_version	1.54.0-SNAPSHOT~11e677e239	1.50.0-SNAPSHOT~f7017caa3c

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1758665748	1758665748
ci_job_id	1144344927	1144344927
ci_pipeline_id	77349259	77349259
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-lgj40339 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-lgj40339 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~f7017caa3c, baseline=1.54.0-SNAPSHOT~11e677e239
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.45 ms) : 2400, 2500
.   : milestone, 2450,
iast (2.186 ms) : 2124, 2248
.   : milestone, 2186,
iast_GLOBAL (2.238 ms) : 2176, 2301
.   : milestone, 2238,
profiling (2.044 ms) : 1994, 2094
.   : milestone, 2044,
tracing (2.014 ms) : 1966, 2063
.   : milestone, 2014,
section candidate
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (3.708 ms) : 3491, 3925
.   : milestone, 3708,
iast (2.188 ms) : 2126, 2250
.   : milestone, 2188,
iast_GLOBAL (2.231 ms) : 2168, 2293
.   : milestone, 2231,
profiling (2.037 ms) : 1986, 2087
.   : milestone, 2037,
tracing (2.025 ms) : 1976, 2073
.   : milestone, 2025,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.464 ms, 1.487 ms]	-
appsec	2.45 ms [2.4 ms, 2.5 ms]	974.342 µs (66.0%)
iast	2.186 ms [2.124 ms, 2.248 ms]	710.005 µs (48.1%)
iast_GLOBAL	2.238 ms [2.176 ms, 2.301 ms]	762.57 µs (51.7%)
profiling	2.044 ms [1.994 ms, 2.094 ms]	568.073 µs (38.5%)
tracing	2.014 ms [1.966 ms, 2.063 ms]	538.495 µs (36.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.463 ms, 1.486 ms]	-
appsec	3.708 ms [3.491 ms, 3.925 ms]	2.234 ms (151.5%)
iast	2.188 ms [2.126 ms, 2.25 ms]	713.535 µs (48.4%)
iast_GLOBAL	2.231 ms [2.168 ms, 2.293 ms]	756.271 µs (51.3%)
profiling	2.037 ms [1.986 ms, 2.087 ms]	562.015 µs (38.1%)
tracing	2.025 ms [1.976 ms, 2.073 ms]	550.016 µs (37.3%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~f7017caa3c, baseline=1.54.0-SNAPSHOT~11e677e239
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.499 s) : 15499000, 15499000
.   : milestone, 15499000,
appsec (15.062 s) : 15062000, 15062000
.   : milestone, 15062000,
iast (18.511 s) : 18511000, 18511000
.   : milestone, 18511000,
iast_GLOBAL (17.881 s) : 17881000, 17881000
.   : milestone, 17881000,
profiling (15.103 s) : 15103000, 15103000
.   : milestone, 15103000,
tracing (14.918 s) : 14918000, 14918000
.   : milestone, 14918000,
section candidate
no_agent (15.602 s) : 15602000, 15602000
.   : milestone, 15602000,
appsec (15.128 s) : 15128000, 15128000
.   : milestone, 15128000,
iast (18.418 s) : 18418000, 18418000
.   : milestone, 18418000,
iast_GLOBAL (17.977 s) : 17977000, 17977000
.   : milestone, 17977000,
profiling (15.607 s) : 15607000, 15607000
.   : milestone, 15607000,
tracing (15.092 s) : 15092000, 15092000
.   : milestone, 15092000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.499 s [15.499 s, 15.499 s]	-
appsec	15.062 s [15.062 s, 15.062 s]	-437.0 ms (-2.8%)
iast	18.511 s [18.511 s, 18.511 s]	3.012 s (19.4%)
iast_GLOBAL	17.881 s [17.881 s, 17.881 s]	2.382 s (15.4%)
profiling	15.103 s [15.103 s, 15.103 s]	-396.0 ms (-2.6%)
tracing	14.918 s [14.918 s, 14.918 s]	-581.0 ms (-3.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.602 s [15.602 s, 15.602 s]	-
appsec	15.128 s [15.128 s, 15.128 s]	-474.0 ms (-3.0%)
iast	18.418 s [18.418 s, 18.418 s]	2.816 s (18.0%)
iast_GLOBAL	17.977 s [17.977 s, 17.977 s]	2.375 s (15.2%)
profiling	15.607 s [15.607 s, 15.607 s]	5.0 ms (0.0%)
tracing	15.092 s [15.092 s, 15.092 s]	-510.0 ms (-3.3%)

Jun 04 '25 13:06 pr-commenter[bot]

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

Jun 05 '25 13:06 github-actions[bot]

I added the do not merge tag for now while we wait for commit-headless to become accessible to public repos and re-discuss whether or not we want to pin system tests.

Sep 23 '25 21:09 sarahchen6

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 59.81% (-0.02%)
View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: f7017ca | Docs | Was this helpful? Give us feedback!}

Sep 23 '25 21:09 datadog-datadog-prod-us1[bot]

Closing this, as the Guild decided we want to stay with the latest version of system-tests.

Oct 15 '25 19:10 sarahchen6

Auto upgrade system-tests

Motivation

What Does This Do

Additional Notes

Contributor Checklist

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary