test: Address App launch times failure test
…ive waits.
Description
This PR aims to put the app launch times tests back in the green. The tests started failing because of a couple of reasons:
- A new onboarding success screen was added to the app. The launch times test did not get updated correctly
- The website we used for testing had UI changes that broke our tests. We will now use the mm test dapp for this test
- The selectors used to identify elements were incorrect.
Note: The warm start tests are failing for a legit reason. The launch times are below the expected bench mark of 8 seconds hovering around ~11 seconds
Related issues
Fixes:
Manual testing steps
- Go to this page...
Screenshots/Recordings
Before
After
Pre-merge author checklist
- [ ] I’ve followed MetaMask Coding Standards.
- [ ] I've completed the PR template to the best of my ability
- [ ] I’ve included tests if applicable
- [ ] I’ve documented my code using JSDoc format if applicable
- [ ] I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.
Pre-merge reviewer checklist
- [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
- [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.
Bitrise
❌❌❌ pr_smoke_e2e_pipeline failed on Bitrise! ❌❌❌
Commit hash: 006e563450a860c4b0e013a30d8b9e70e72dc910 Build link: https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/79ff495c-229d-41a7-a2f6-b5702e8da0ed
[!NOTE]
- You can kick off another
pr_smoke_e2e_pipelineon Bitrise by removing and re-applying theRun Smoke E2Elabel on the pull request
One smoke test is failing
There seems to be a latency issue with the app? This test does not touch anything swaps related. I will investigate on main and see.
Let's we also post a launch time test run in the comments to confirm the fix
Ah, so this is what I was discussing during the stand-up. The point of failure in the tests has been fixed, but the tests still do not pass the 8-second benchmark. App code changes are needed to address the performance issue.
. See Tomas's message here: https://consensys.slack.com/archives/C04GLB755TK/p1716311340155619
@cortisiko Could you post a link of the successful app start time test in Bitrise?
@Cal-L Because this test relies on changes to the app code to improve performance I went ahead and updated the bench mark from 8 seconds to 12 seconds.
Here is a screenshot of passing warm start tests
I will kick off a bitrise build right now
Quality Gate passed
Issues
0 New issues
0 Accepted issues
Measures
0 Security Hotspots
No data about Coverage
No data about Duplication
Bitrise
✅✅✅ pr_smoke_e2e_pipeline passed on Bitrise! ✅✅✅
Commit hash: 2695475b81c6f4008faea63774bdda36c0833ca8 Build link: https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/fd14331a-f2b6-4f68-a66e-f537f2368ad8
[!NOTE]
- You can kick off another
pr_smoke_e2e_pipelineon Bitrise by removing and re-applying theRun Smoke E2Elabel on the pull request
@Cal-L Passing App launch times test on bitrise with adjusted benchmark: https://app.bitrise.io/build/4bb0f921-edf8-4685-a30d-0e2843819686