sentry-java
sentry-java copied to clipboard
Add New UI Tests App for critical tests
:scroll: Description
This PR adds crash recovery and corrupted envelope recovery UI Tests for Sentry Android.
:bulb: Motivation and Context
- Part of https://github.com/getsentry/projects/issues/165
:green_heart: How did you test it?
locally, ci
:pencil: Checklist
- [ ] I reviewed the submitted code.
- [ ] I added tests to verify the changes.
- [ ] No new PII added or SDK only sends newly added PII if
sendDefaultPIIis enabled. - [ ] I updated the docs if needed.
- [ ] Review from the native team if needed.
- [ ] No breaking change or entry added to the changelog.
- [ ] No breaking change for hybrid SDKs or communicated to hybrid SDKs.
:crystal_ball: Todo
- [ ] Add workflow
- [ ] Add make command for local execution
Performance metrics :rocket:
| Plain | With Sentry | Diff | |
|---|---|---|---|
| Startup time | 491.93 ms | 568.02 ms | 76.09 ms |
| Size | 1.70 MiB | 2.35 MiB | 667.95 KiB |
Previous results on branch: kw/test/add-critical-ui-tests
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| c2743c4cbd0c60dfaa11087b48de96976e8723f2 | 427.73 ms | 439.66 ms | 11.93 ms |
| d726180eca7e12811624974f887f16ce2eb5d9d3 | 388.76 ms | 414.46 ms | 25.70 ms |
| ffd2c3c5527a6efb94e5c7007fa58c2039795442 | 405.94 ms | 419.46 ms | 13.52 ms |
| 8e916549ccebacbd74f94907f7fc59d270e7f9d8 | 399.11 ms | 458.86 ms | 59.75 ms |
| 4e066a140530958ade3315a0cd00d5a196709d30 | 401.74 ms | 460.76 ms | 59.02 ms |
| 34446d9234eea96c4dbf45ef90cdf7cae95c5762 | 408.82 ms | 415.42 ms | 6.60 ms |
| b4967a1003603607a8571d8343d1c642514b26ca | 437.10 ms | 492.22 ms | 55.12 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| c2743c4cbd0c60dfaa11087b48de96976e8723f2 | 1.70 MiB | 2.35 MiB | 667.95 KiB |
| d726180eca7e12811624974f887f16ce2eb5d9d3 | 1.70 MiB | 2.35 MiB | 667.95 KiB |
| ffd2c3c5527a6efb94e5c7007fa58c2039795442 | 1.70 MiB | 2.35 MiB | 668.15 KiB |
| 8e916549ccebacbd74f94907f7fc59d270e7f9d8 | 1.70 MiB | 2.35 MiB | 667.95 KiB |
| 4e066a140530958ade3315a0cd00d5a196709d30 | 1.70 MiB | 2.35 MiB | 667.95 KiB |
| 34446d9234eea96c4dbf45ef90cdf7cae95c5762 | 1.70 MiB | 2.35 MiB | 667.95 KiB |
| b4967a1003603607a8571d8343d1c642514b26ca | 1.70 MiB | 2.35 MiB | 667.95 KiB |
I'm currently waiting for the job to run multiple times (50-100x) to ensure it's stable, before merging.
Run from stability testing https://github.com/getsentry/sentry-java/actions/runs/11439418099/job/31828821106?pr=3802 (27 runs, 1 emulator boot timeout)