test: speed up DI integration tests even more
What does this PR do?
Don't wait for script loading to stabilize in Dynamic Instrumentation integration tests.
This might make them temporarily set the test breakpoint in the wrong location, but it will quickly get moved to the right location and the test should still trigger as expected.
Debugger CI run performance improvement:
| Before | After |
|---|---|
Motivation
Plugin Checklist
- [ ] Unit tests.
- [ ] Integration tests.
- [ ] Benchmarks.
- [ ] TypeScript definitions.
- [ ] TypeScript tests.
- [ ] API documentation.
- [ ] CI jobs/workflows.
Additional Notes
-
#5864
👈 (View in Graphite)
-
#5863
-
#5861
-
master
This stack of pull requests is managed by Graphite. Learn more about stacking.
Overall package size
Self size: 9.63 MB Deduped: 104.57 MB No deduping: 105.09 MB
Dependency sizes
| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.6.0 | 30.47 MB | 30.47 MB | | @datadog/native-appsec | 8.5.2 | 19.33 MB | 19.34 MB | | @datadog/pprof | 5.8.0 | 12.55 MB | 12.92 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.3 | 2.95 MB | 5.6 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.14.0 | 120.58 kB | 841.68 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | dc-polyfill | 0.1.9 | 25.11 kB | 25.11 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.2 | 23.54 kB | 23.54 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 80.75%. Comparing base (
784197d) to head (a73650f). Report is 78 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #5864 +/- ##
=======================================
Coverage 80.75% 80.75%
=======================================
Files 464 464
Lines 19910 19910
=======================================
Hits 16078 16078
Misses 3832 3832
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
Benchmarks
Benchmark execution time: 2025-06-10 19:44:23
Comparing candidate commit a73650f2d9718358d553a19b025ef022ef6505d7 in PR branch watson/speed-up-di-tests2 with baseline commit 784197db4d3c3254ad46e8e10a2d79a8524970dd in branch master.
Found 0 performance improvements and 0 performance regressions! Performance is the same for 1267 metrics, 56 unstable metrics.
Datadog Report
Branch report: watson/speed-up-di-tests2
Commit report: e699f99
Test service: dd-trace-js-integration-tests
:white_check_mark: 0 Failed, 1086 Passed, 0 Skipped, 15m 0.62s Total Time :snowflake: 1 New Flaky
New Flaky Tests (1)
-
[email protected] commonJS flaky test retries retries flaky tests-integration-tests/cypress/cypress.spec.js- Last FailureExpand for error
xpected +0 to equal 1
Actually what does this even do exactly? According to the description it seems like the right breakpoint will eventually happen anyway, so why have the delay to begin with?
Any way to do this without creating such a variable?
I haven't been able to think of anything, as it has to be configurable from outside the sandbox 🤷
Actually what does this even do exactly? According to the description it seems like the right breakpoint will eventually happen anyway, so why have the delay to begin with?
It's to avoid the false positive. Both the data collected would not be what was expected and confuse users, and the incorrectly set breakpoint might be in a hot code-path, which could be a big problem (worst case scenario)
Seems to me like a design flaw more than something that should be configurable. I'm going to agree with @BridgeAR here. This is equivalent to adding a sleep which wouldn't fix the issue but only make it less likely, and I think we should figure out an actual fix for the underlying issue instead.
Haha, yeah I knew this change was going to be controversial - and to be honest I don't really like it myself either. But I'm really happy for the discussion. I'll mark this as draft for now and think a bit more about it. Also, it's not super important to get this merged. It would just be good DevEx ☺️
I've decided it's not worth it, as it's better to test closer to the real world experience