dd-trace-java icon indicating copy to clipboard operation
dd-trace-java copied to clipboard

Fix Exception Replay in Lambda

Open nhulston opened this issue 8 months ago • 2 comments

What Does This Do

In Lambda, the inner stack trace is truncated -- stack frames from fileName='AWSLambda.java' are removed. Therefore, in the original code, we would end up with a negative currentIdx, resulting in an error occurring in sanityCheckSnapshotAssignment.

This change adds a simple fallback:

      int currentIdx = innerTrace.length - snapshot.getStack().size();

      if (currentIdx < 0) {
        // This means the innerTrace was truncated by the underlying environment.
        // This is known to happen in AWS Lambda, but may also happen elsewhere.
        currentIdx = i;
      }

From manual testing in Lambda, this correctly gets the index in Lambda for all snapshots. Screenshot 2025-05-19 at 1 21 16 PM

Motivation

https://datadoghq.atlassian.net/browse/SVLS-6855

Additional Notes

  • Exception Replay requires the exception to occur twice in Java: the first occurrence transforms the class, and the second occurrence captures the local variables
  • In Java, currently method parameters are captured by Exception Replay but other local variables are not captured yet

Contributor Checklist

Jira ticket: [PROJ-IDENT]

nhulston avatar May 19 '25 17:05 nhulston

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1747845985 1747846355
end_time 2025-05-21T16:47:46 2025-05-21T16:53:56
git_branch master nicholas.hulston/fix-exception-replay-in-lambda
git_commit_sha 58089f324f 1f12c25da2090e81cf62067eff67bf00cd0e1eda
start_time 2025-05-21T16:46:26 2025-05-21T16:52:36
See matching parameters
Baseline Candidate
ci_job_id 948550027 948550027
ci_pipeline_id 65761122 65761122
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1747845400 1747845400

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 4 metrics, 6 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+579.873µs; +614.558µs] or [+5.767%; +6.112%]
worse
[+594.154µs; +651.668µs] or [+5.824%; +6.387%]
worse
[+553.132µs; +651.974µs] or [+5.373%; +6.333%]
worse
[+500.061µs; +843.173µs] or [+4.727%; +7.971%]
worse
[-6.440op/s; -4.149op/s] or [-6.665%; -4.294%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-30.568µs; +43.765µs] or [-10.925%; +15.642%]
unstable
[-44.257µs; +58.112µs] or [-13.640%; +17.910%]
unstable
[-54.772µs; +68.325µs] or [-16.113%; +20.100%]
unstable
[-140.565µs; +409.950µs] or [-14.919%; +43.509%]
same
scenario:basic same same same unstable
[-38.673µs; +141.429µs] or [-4.807%; +17.581%]
unstable
[-231.458op/s; +115.314op/s] or [-9.490%; +4.728%]
Request duration reports for reports
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (324.475 µs) : 284, 365
.   : milestone, 324,
basic (315.173 µs) : 305, 325
.   : milestone, 315,
loop (10.203 ms) : 10172, 10233
.   : milestone, 10203,
section candidate
noprobe (331.402 µs) : 276, 387
.   : milestone, 331,
basic (316.213 µs) : 306, 326
.   : milestone, 316,
loop (10.826 ms) : 10804, 10848
.   : milestone, 10826,
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 324.475 µs [284.051 µs, 364.899 µs]
basic 315.173 µs [305.194 µs, 325.151 µs]
loop 10.203 ms [10.172 ms, 10.233 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 331.402 µs [275.595 µs, 387.21 µs]
basic 316.213 µs [306.048 µs, 326.377 µs]
loop 10.826 ms [10.804 ms, 10.848 ms]

pr-commenter[bot] avatar May 19 '25 17:05 pr-commenter[bot]

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nicholas.hulston/fix-exception-replay-in-lambda
git_commit_date 1747843428 1747845400
git_commit_sha 58089f324f 1f12c25da2
release_version 1.50.0-SNAPSHOT~58089f324f 1.50.0-SNAPSHOT~1f12c25da2
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1747847720 1747847720
ci_job_id 948550021 948550021
ci_pipeline_id 65761122 65761122
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-pw3vcngw-project-304-concurrent-2-mr4c9knq 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-pw3vcngw-project-304-concurrent-2-mr4c9knq 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 56 metrics, 14 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:Remote Config worse
[+14.368µs; +59.830µs] or [+2.487%; +10.356%]
614.835µs 577.736µs
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.026 s) : 0, 1026092
Total [baseline] (10.455 s) : 0, 10455475
Agent [candidate] (1.022 s) : 0, 1021593
Total [candidate] (10.455 s) : 0, 10455242
section appsec
Agent [baseline] (1.16 s) : 0, 1160093
Total [baseline] (10.639 s) : 0, 10638653
Agent [candidate] (1.16 s) : 0, 1159737
Total [candidate] (10.628 s) : 0, 10628407
section iast
Agent [baseline] (1.165 s) : 0, 1164850
Total [baseline] (10.925 s) : 0, 10924915
Agent [candidate] (1.147 s) : 0, 1147400
Total [candidate] (10.93 s) : 0, 10930161
section profiling
Agent [baseline] (1.266 s) : 0, 1265807
Total [baseline] (10.816 s) : 0, 10816311
Agent [candidate] (1.273 s) : 0, 1272961
Total [candidate] (10.86 s) : 0, 10860176
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.026 s -
Agent appsec 1.16 s 134.001 ms (13.1%)
Agent iast 1.165 s 138.757 ms (13.5%)
Agent profiling 1.266 s 239.715 ms (23.4%)
Total tracing 10.455 s -
Total appsec 10.639 s 183.178 ms (1.8%)
Total iast 10.925 s 469.44 ms (4.5%)
Total profiling 10.816 s 360.836 ms (3.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.022 s -
Agent appsec 1.16 s 138.145 ms (13.5%)
Agent iast 1.147 s 125.808 ms (12.3%)
Agent profiling 1.273 s 251.369 ms (24.6%)
Total tracing 10.455 s -
Total appsec 10.628 s 173.165 ms (1.7%)
Total iast 10.93 s 474.919 ms (4.5%)
Total profiling 10.86 s 404.934 ms (3.9%)
gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (684.611 ms) : 0, 684611
BytebuddyAgent [candidate] (682.427 ms) : 0, 682427
GlobalTracer [baseline] (242.585 ms) : 0, 242585
GlobalTracer [candidate] (240.721 ms) : 0, 240721
AppSec [baseline] (55.616 ms) : 0, 55616
AppSec [candidate] (55.173 ms) : 0, 55173
Debugger [baseline] (7.611 ms) : 0, 7611
Debugger [candidate] (6.857 ms) : 0, 6857
Remote Config [baseline] (697.175 µs) : 0, 697
Remote Config [candidate] (702.882 µs) : 0, 703
Telemetry [baseline] (11.34 ms) : 0, 11340
Telemetry [candidate] (12.116 ms) : 0, 12116
section appsec
BytebuddyAgent [baseline] (699.866 ms) : 0, 699866
BytebuddyAgent [candidate] (699.86 ms) : 0, 699860
GlobalTracer [baseline] (236.36 ms) : 0, 236360
GlobalTracer [candidate] (236.605 ms) : 0, 236605
AppSec [baseline] (175.114 ms) : 0, 175114
AppSec [candidate] (175.007 ms) : 0, 175007
Debugger [baseline] (5.938 ms) : 0, 5938
Debugger [candidate] (5.899 ms) : 0, 5899
Remote Config [baseline] (630.746 µs) : 0, 631
Remote Config [candidate] (634.404 µs) : 0, 634
Telemetry [baseline] (7.779 ms) : 0, 7779
Telemetry [candidate] (7.372 ms) : 0, 7372
IAST [baseline] (21.73 ms) : 0, 21730
IAST [candidate] (21.753 ms) : 0, 21753
section iast
BytebuddyAgent [baseline] (814.416 ms) : 0, 814416
BytebuddyAgent [candidate] (801.041 ms) : 0, 801041
GlobalTracer [baseline] (233.097 ms) : 0, 233097
GlobalTracer [candidate] (230.372 ms) : 0, 230372
AppSec [baseline] (53.004 ms) : 0, 53004
AppSec [candidate] (51.114 ms) : 0, 51114
Debugger [baseline] (5.918 ms) : 0, 5918
Debugger [candidate] (5.779 ms) : 0, 5779
Remote Config [baseline] (593.746 µs) : 0, 594
Remote Config [candidate] (577.072 µs) : 0, 577
Telemetry [baseline] (7.861 ms) : 0, 7861
Telemetry [candidate] (7.799 ms) : 0, 7799
IAST [baseline] (26.197 ms) : 0, 26197
IAST [candidate] (26.493 ms) : 0, 26493
section profiling
BytebuddyAgent [baseline] (675.572 ms) : 0, 675572
BytebuddyAgent [candidate] (678.634 ms) : 0, 678634
GlobalTracer [baseline] (359.634 ms) : 0, 359634
GlobalTracer [candidate] (362.027 ms) : 0, 362027
AppSec [baseline] (61.801 ms) : 0, 61801
AppSec [candidate] (62.264 ms) : 0, 62264
Debugger [baseline] (6.251 ms) : 0, 6251
Debugger [candidate] (6.36 ms) : 0, 6360
Remote Config [baseline] (659.852 µs) : 0, 660
Remote Config [candidate] (657.077 µs) : 0, 657
Telemetry [baseline] (8.195 ms) : 0, 8195
Telemetry [candidate] (8.313 ms) : 0, 8313
ProfilingAgent [baseline] (102.814 ms) : 0, 102814
ProfilingAgent [candidate] (103.589 ms) : 0, 103589
Profiling [baseline] (102.838 ms) : 0, 102838
Profiling [candidate] (103.613 ms) : 0, 103613
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.021 s) : 0, 1020732
Total [baseline] (8.652 s) : 0, 8652168
Agent [candidate] (1.017 s) : 0, 1017118
Total [candidate] (8.607 s) : 0, 8607095
section iast
Agent [baseline] (1.15 s) : 0, 1149518
Total [baseline] (9.228 s) : 0, 9228251
Agent [candidate] (1.148 s) : 0, 1147968
Total [candidate] (9.228 s) : 0, 9228182
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.151 s) : 0, 1151419
Total [baseline] (9.221 s) : 0, 9220792
Agent [candidate] (1.15 s) : 0, 1150253
Total [candidate] (9.199 s) : 0, 9199083
section iast_TELEMETRY_OFF
Agent [baseline] (1.145 s) : 0, 1145382
Total [baseline] (9.193 s) : 0, 9192773
Agent [candidate] (1.155 s) : 0, 1154517
Total [candidate] (9.232 s) : 0, 9232212
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.021 s -
Agent iast 1.15 s 128.786 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.151 s 130.687 ms (12.8%)
Agent iast_TELEMETRY_OFF 1.145 s 124.65 ms (12.2%)
Total tracing 8.652 s -
Total iast 9.228 s 576.083 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.221 s 568.624 ms (6.6%)
Total iast_TELEMETRY_OFF 9.193 s 540.605 ms (6.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.017 s -
Agent iast 1.148 s 130.85 ms (12.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.15 s 133.136 ms (13.1%)
Agent iast_TELEMETRY_OFF 1.155 s 137.4 ms (13.5%)
Total tracing 8.607 s -
Total iast 9.228 s 621.086 ms (7.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.199 s 591.988 ms (6.9%)
Total iast_TELEMETRY_OFF 9.232 s 625.117 ms (7.3%)
gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.628 ms) : 0, 682628
BytebuddyAgent [candidate] (680.3 ms) : 0, 680300
GlobalTracer [baseline] (240.517 ms) : 0, 240517
GlobalTracer [candidate] (239.421 ms) : 0, 239421
AppSec [baseline] (54.458 ms) : 0, 54458
AppSec [candidate] (54.329 ms) : 0, 54329
Debugger [baseline] (9.079 ms) : 0, 9079
Debugger [candidate] (8.968 ms) : 0, 8968
Remote Config [baseline] (700.994 µs) : 0, 701
Remote Config [candidate] (683.736 µs) : 0, 684
Telemetry [baseline] (9.807 ms) : 0, 9807
Telemetry [candidate] (9.797 ms) : 0, 9797
section iast
BytebuddyAgent [baseline] (802.082 ms) : 0, 802082
BytebuddyAgent [candidate] (800.67 ms) : 0, 800670
GlobalTracer [baseline] (230.996 ms) : 0, 230996
GlobalTracer [candidate] (230.939 ms) : 0, 230939
AppSec [baseline] (50.269 ms) : 0, 50269
AppSec [candidate] (49.5 ms) : 0, 49500
Debugger [baseline] (5.907 ms) : 0, 5907
Debugger [candidate] (5.919 ms) : 0, 5919
Remote Config [baseline] (577.736 µs) : 0, 578
Remote Config [candidate] (614.835 µs) : 0, 615
Telemetry [baseline] (7.883 ms) : 0, 7883
Telemetry [candidate] (7.859 ms) : 0, 7859
IAST [baseline] (27.544 ms) : 0, 27544
IAST [candidate] (28.999 ms) : 0, 28999
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (801.813 ms) : 0, 801813
BytebuddyAgent [candidate] (802.5 ms) : 0, 802500
GlobalTracer [baseline] (231.424 ms) : 0, 231424
GlobalTracer [candidate] (231.166 ms) : 0, 231166
AppSec [baseline] (52.889 ms) : 0, 52889
AppSec [candidate] (51.149 ms) : 0, 51149
Debugger [baseline] (5.995 ms) : 0, 5995
Debugger [candidate] (5.959 ms) : 0, 5959
Remote Config [baseline] (628.716 µs) : 0, 629
Remote Config [candidate] (595.533 µs) : 0, 596
Telemetry [baseline] (7.972 ms) : 0, 7972
Telemetry [candidate] (7.965 ms) : 0, 7965
IAST [baseline] (27.275 ms) : 0, 27275
IAST [candidate] (27.387 ms) : 0, 27387
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (799.26 ms) : 0, 799260
BytebuddyAgent [candidate] (805.686 ms) : 0, 805686
GlobalTracer [baseline] (230.174 ms) : 0, 230174
GlobalTracer [candidate] (231.77 ms) : 0, 231770
AppSec [baseline] (54.287 ms) : 0, 54287
AppSec [candidate] (55.681 ms) : 0, 55681
Debugger [baseline] (5.957 ms) : 0, 5957
Debugger [candidate] (6.005 ms) : 0, 6005
Remote Config [baseline] (587.194 µs) : 0, 587
Remote Config [candidate] (623.31 µs) : 0, 623
Telemetry [baseline] (7.755 ms) : 0, 7755
Telemetry [candidate] (7.804 ms) : 0, 7804
IAST [baseline] (23.809 ms) : 0, 23809
IAST [candidate] (23.242 ms) : 0, 23242

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-05-21T16:47:30 2025-05-21T16:55:15
git_branch master nicholas.hulston/fix-exception-replay-in-lambda
git_commit_date 1747843428 1747845400
git_commit_sha 58089f324f 1f12c25da2
release_version 1.50.0-SNAPSHOT~58089f324f 1.50.0-SNAPSHOT~1f12c25da2
start_time 2025-05-21T16:47:16 2025-05-21T16:55:00
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1747846912 1747846912
ci_job_id 948550022 948550022
ci_pipeline_id 65761122 65761122
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-3qkfhwhr-project-304-concurrent-0-ex7sp2gz 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-3qkfhwhr-project-304-concurrent-0-ex7sp2gz 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.366 ms) : 1345, 1386
.   : milestone, 1366,
appsec (1.731 ms) : 1709, 1754
.   : milestone, 1731,
appsec_no_iast (1.735 ms) : 1712, 1758
.   : milestone, 1735,
code_origins (1.677 ms) : 1649, 1704
.   : milestone, 1677,
iast (1.505 ms) : 1480, 1529
.   : milestone, 1505,
profiling (1.516 ms) : 1492, 1540
.   : milestone, 1516,
tracing (1.498 ms) : 1473, 1523
.   : milestone, 1498,
section candidate
no_agent (1.345 ms) : 1326, 1364
.   : milestone, 1345,
appsec (1.729 ms) : 1705, 1752
.   : milestone, 1729,
appsec_no_iast (1.722 ms) : 1698, 1746
.   : milestone, 1722,
code_origins (1.661 ms) : 1634, 1687
.   : milestone, 1661,
iast (1.519 ms) : 1495, 1542
.   : milestone, 1519,
profiling (1.526 ms) : 1501, 1551
.   : milestone, 1526,
tracing (1.496 ms) : 1474, 1519
.   : milestone, 1496,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.366 ms [1.345 ms, 1.386 ms] -
appsec 1.731 ms [1.709 ms, 1.754 ms] 365.75 µs (26.8%)
appsec_no_iast 1.735 ms [1.712 ms, 1.758 ms] 369.749 µs (27.1%)
code_origins 1.677 ms [1.649 ms, 1.704 ms] 310.987 µs (22.8%)
iast 1.505 ms [1.48 ms, 1.529 ms] 139.004 µs (10.2%)
profiling 1.516 ms [1.492 ms, 1.54 ms] 150.313 µs (11.0%)
tracing 1.498 ms [1.473 ms, 1.523 ms] 132.232 µs (9.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.345 ms [1.326 ms, 1.364 ms] -
appsec 1.729 ms [1.705 ms, 1.752 ms] 383.554 µs (28.5%)
appsec_no_iast 1.722 ms [1.698 ms, 1.746 ms] 376.675 µs (28.0%)
code_origins 1.661 ms [1.634 ms, 1.687 ms] 315.461 µs (23.5%)
iast 1.519 ms [1.495 ms, 1.542 ms] 173.606 µs (12.9%)
profiling 1.526 ms [1.501 ms, 1.551 ms] 180.613 µs (13.4%)
tracing 1.496 ms [1.474 ms, 1.519 ms] 151.12 µs (11.2%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f
    dateFormat X
    axisFormat %s
section baseline
no_agent (380.8 µs) : 361, 401
.   : milestone, 381,
iast (522.769 µs) : 501, 545
.   : milestone, 523,
iast_FULL (738.32 µs) : 715, 762
.   : milestone, 738,
iast_GLOBAL (569.882 µs) : 547, 592
.   : milestone, 570,
iast_HARDCODED_SECRET_DISABLED (521.483 µs) : 499, 544
.   : milestone, 521,
iast_INACTIVE (460.825 µs) : 440, 482
.   : milestone, 461,
iast_TELEMETRY_OFF (515.777 µs) : 493, 539
.   : milestone, 516,
tracing (464.261 µs) : 442, 487
.   : milestone, 464,
section candidate
no_agent (378.1 µs) : 358, 398
.   : milestone, 378,
iast (523.531 µs) : 500, 547
.   : milestone, 524,
iast_FULL (730.646 µs) : 709, 753
.   : milestone, 731,
iast_GLOBAL (573.889 µs) : 551, 597
.   : milestone, 574,
iast_HARDCODED_SECRET_DISABLED (518.315 µs) : 495, 542
.   : milestone, 518,
iast_INACTIVE (463.142 µs) : 441, 485
.   : milestone, 463,
iast_TELEMETRY_OFF (517.545 µs) : 494, 541
.   : milestone, 518,
tracing (459.539 µs) : 437, 482
.   : milestone, 460,
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 380.8 µs [360.861 µs, 400.739 µs] -
iast 522.769 µs [501.016 µs, 544.523 µs] 141.969 µs (37.3%)
iast_FULL 738.32 µs [714.793 µs, 761.847 µs] 357.52 µs (93.9%)
iast_GLOBAL 569.882 µs [547.362 µs, 592.401 µs] 189.082 µs (49.7%)
iast_HARDCODED_SECRET_DISABLED 521.483 µs [498.989 µs, 543.977 µs] 140.682 µs (36.9%)
iast_INACTIVE 460.825 µs [439.525 µs, 482.124 µs] 80.024 µs (21.0%)
iast_TELEMETRY_OFF 515.777 µs [492.548 µs, 539.006 µs] 134.977 µs (35.4%)
tracing 464.261 µs [441.753 µs, 486.769 µs] 83.461 µs (21.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.1 µs [358.202 µs, 397.999 µs] -
iast 523.531 µs [500.093 µs, 546.969 µs] 145.431 µs (38.5%)
iast_FULL 730.646 µs [708.757 µs, 752.536 µs] 352.546 µs (93.2%)
iast_GLOBAL 573.889 µs [550.687 µs, 597.09 µs] 195.788 µs (51.8%)
iast_HARDCODED_SECRET_DISABLED 518.315 µs [495.1 µs, 541.53 µs] 140.214 µs (37.1%)
iast_INACTIVE 463.142 µs [440.916 µs, 485.368 µs] 85.041 µs (22.5%)
iast_TELEMETRY_OFF 517.545 µs [493.977 µs, 541.114 µs] 139.445 µs (36.9%)
tracing 459.539 µs [436.908 µs, 482.169 µs] 81.438 µs (21.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nicholas.hulston/fix-exception-replay-in-lambda
git_commit_date 1747843428 1747845400
git_commit_sha 58089f324f 1f12c25da2
release_version 1.50.0-SNAPSHOT~58089f324f 1.50.0-SNAPSHOT~1f12c25da2
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1747847409 1747847409
ci_job_id 948550023 948550023
ci_pipeline_id 65761122 65761122
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-pw3vcngw-project-304-concurrent-3-huawmceg 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-pw3vcngw-project-304-concurrent-3-huawmceg 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (2.394 ms) : 2345, 2442
.   : milestone, 2394,
iast (2.174 ms) : 2113, 2235
.   : milestone, 2174,
iast_GLOBAL (2.211 ms) : 2150, 2272
.   : milestone, 2211,
profiling (2.032 ms) : 1982, 2082
.   : milestone, 2032,
tracing (2.004 ms) : 1956, 2051
.   : milestone, 2004,
section candidate
no_agent (1.478 ms) : 1467, 1490
.   : milestone, 1478,
appsec (2.388 ms) : 2340, 2437
.   : milestone, 2388,
iast (2.175 ms) : 2114, 2237
.   : milestone, 2175,
iast_GLOBAL (2.212 ms) : 2151, 2273
.   : milestone, 2212,
profiling (2.028 ms) : 1979, 2077
.   : milestone, 2028,
tracing (1.998 ms) : 1951, 2046
.   : milestone, 1998,
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.462 ms, 1.485 ms] -
appsec 2.394 ms [2.345 ms, 2.442 ms] 920.421 µs (62.5%)
iast 2.174 ms [2.113 ms, 2.235 ms] 700.915 µs (47.6%)
iast_GLOBAL 2.211 ms [2.15 ms, 2.272 ms] 737.525 µs (50.1%)
profiling 2.032 ms [1.982 ms, 2.082 ms] 558.637 µs (37.9%)
tracing 2.004 ms [1.956 ms, 2.051 ms] 530.355 µs (36.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.478 ms [1.467 ms, 1.49 ms] -
appsec 2.388 ms [2.34 ms, 2.437 ms] 910.161 µs (61.6%)
iast 2.175 ms [2.114 ms, 2.237 ms] 697.035 µs (47.2%)
iast_GLOBAL 2.212 ms [2.151 ms, 2.273 ms] 733.614 µs (49.6%)
profiling 2.028 ms [1.979 ms, 2.077 ms] 549.884 µs (37.2%)
tracing 1.998 ms [1.951 ms, 2.046 ms] 520.198 µs (35.2%)
Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.961 s) : 14961000, 14961000
.   : milestone, 14961000,
appsec (14.851 s) : 14851000, 14851000
.   : milestone, 14851000,
iast (18.796 s) : 18796000, 18796000
.   : milestone, 18796000,
iast_GLOBAL (18.328 s) : 18328000, 18328000
.   : milestone, 18328000,
profiling (15.29 s) : 15290000, 15290000
.   : milestone, 15290000,
tracing (14.993 s) : 14993000, 14993000
.   : milestone, 14993000,
section candidate
no_agent (15.291 s) : 15291000, 15291000
.   : milestone, 15291000,
appsec (14.893 s) : 14893000, 14893000
.   : milestone, 14893000,
iast (18.725 s) : 18725000, 18725000
.   : milestone, 18725000,
iast_GLOBAL (17.728 s) : 17728000, 17728000
.   : milestone, 17728000,
profiling (15.106 s) : 15106000, 15106000
.   : milestone, 15106000,
tracing (14.71 s) : 14710000, 14710000
.   : milestone, 14710000,
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.961 s [14.961 s, 14.961 s] -
appsec 14.851 s [14.851 s, 14.851 s] -110.0 ms (-0.7%)
iast 18.796 s [18.796 s, 18.796 s] 3.835 s (25.6%)
iast_GLOBAL 18.328 s [18.328 s, 18.328 s] 3.367 s (22.5%)
profiling 15.29 s [15.29 s, 15.29 s] 329.0 ms (2.2%)
tracing 14.993 s [14.993 s, 14.993 s] 32.0 ms (0.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.291 s [15.291 s, 15.291 s] -
appsec 14.893 s [14.893 s, 14.893 s] -398.0 ms (-2.6%)
iast 18.725 s [18.725 s, 18.725 s] 3.434 s (22.5%)
iast_GLOBAL 17.728 s [17.728 s, 17.728 s] 2.437 s (15.9%)
profiling 15.106 s [15.106 s, 15.106 s] -185.0 ms (-1.2%)
tracing 14.71 s [14.71 s, 14.71 s] -581.0 ms (-3.8%)

pr-commenter[bot] avatar May 19 '25 17:05 pr-commenter[bot]

This pull request has been marked as stale because it has not had activity over the past quarter. It will be closed in 7 days if no further activity occurs. Feel free to reopen the PR if you are still working on it.

github-actions[bot] avatar Nov 01 '25 03:11 github-actions[bot]

This pull request has been closed because it has not had activity over the past quarter. Feel free to reopen the PR if you are still working on it.

github-actions[bot] avatar Dec 01 '25 03:12 github-actions[bot]