Fix navigation e2e tests
Related issues
I noticed that after merging #2054, a couple of tests were still flaky on trunk. This PR fixes the issues reported on these runs: 39bbd-pb
Proposed Changes
- Increases the timeout for the "View Post" button after post creation
- Attempts to close the welcome guide modal more times
- Adds a wait for networkidle after media upload
Testing Instructions
- CI should be green
- Visual review
Pre-merge Checklist
- [ ] Have you checked for TypeScript, React or other console errors?
📊 Performance Test Results
Comparing a8a479f36ae5235ac9b8d28b85acd83c6bd7436c vs trunk
site-editor
| Metric | trunk | a8a479f36ae5235ac9b8d28b85acd83c6bd7436c | Diff | Change |
|---|---|---|---|---|
| load | 8212.00 ms | 9479.00 ms | +1267.00 ms | 🔴 15.4% |
site-startup
| Metric | trunk | a8a479f36ae5235ac9b8d28b85acd83c6bd7436c | Diff | Change |
|---|---|---|---|---|
| siteCreation | 18223.00 ms | 15063.00 ms | -3160.00 ms | 🟢 -17.3% |
| siteStartup | 5950.00 ms | 5956.00 ms | +6.00 ms | 🔴 0.1% |
Results are median values from multiple test runs.
Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change
I tested it locally and for me the test
creates a postfails. It seems that it correctly clicks the Publish button, but then it automatically clicks the "View post" button even before the post has been saved, so when the posts load, the new post doesn't appear yet.e2e-view-post-error.mp4 I tested with these two commands:
npx playwright test --grep "Site Navigation" --uiand
❯ npm run e2e -- site-navigation.test.ts > [email protected] e2e > npx playwright install && npx playwright test site-navigation.test.ts Running 9 tests using 1 worker ✓ 1 e2e/site-navigation.test.ts:77:6 › Site Navigation › opens site at homepage (626ms) ✓ 2 e2e/site-navigation.test.ts:94:6 › Site Navigation › opens and automatically logs in to WP Admin (4.9s) ✘ 3 e2e/site-navigation.test.ts:110:6 › Site Navigation › creates a post (21.6s) ✓ 4 e2e/site-navigation.test.ts:156:6 › Site Navigation › uploads media (9.3s) ✓ 5 e2e/site-navigation.test.ts:192:6 › Site Navigation › activates themes (2.4s) ✓ 6 e2e/site-navigation.test.ts:218:6 › Site Navigation › adds new themes (17.6s) ✓ 7 e2e/site-navigation.test.ts:247:6 › Site Navigation › activates plugin (3.0s) ✓ 8 e2e/site-navigation.test.ts:269:6 › Site Navigation › adds new plugin (18.3s) ✓ 9 e2e/site-navigation.test.ts:299:6 › Site Navigation › "Post name" permalink structure works (13.6s) 1) e2e/site-navigation.test.ts:110:6 › Site Navigation › creates a post ────────────────────────── Error: Timed out 10000ms waiting for expect(locator).toBeVisible() Locator: locator('a.row-title:has-text("E2E Test Post")') Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 10000ms - waiting for locator('a.row-title:has-text("E2E Test Post")') 151 | // Verify post was created by visiting posts list 152 | await page.goto( getUrlWithAutoLogin( `${ wpAdminUrl }/edit.php` ) ); > 153 | await expect( page.locator( 'a.row-title:has-text("E2E Test Post")' ) ).toBeVisible(); | ^ 154 | } ); 155 | 156 | test( 'uploads media', async ( { page } ) => { at /Users/macbookpro/Documents/projects-m3.nosync/studio/e2e/site-navigation.test.ts:153:75 Error Context: test-results/site-navigation-Site-Navigation-creates-a-post/error-context.md attachment #2: trace (application/zip) ───────────────────────────────────────────────────────── test-results/site-navigation-Site-Navigation-creates-a-post/trace.zip Usage: npx playwright show-trace test-results/site-navigation-Site-Navigation-creates-a-post/trace.zip ──────────────────────────────────────────────────────────────────────────────────────────────── 1 failed e2e/site-navigation.test.ts:110:6 › Site Navigation › creates a post ─────────────────────────── 8 passed (2.3m)
Thanks @sejas for the sharing the detailed logs, I was occasionally getting this error but thought that the most recent changes fixed it. 🤔 The CI also failed on a different issue again where the Welcome modal obstructs the view:
2) e2e/site-navigation.test.ts:301:6 › Site Navigation › "Post name" permalink structure works ───
--
Test timeout of 180000ms exceeded.
Error: locator.click: Test timeout of 180000ms exceeded.
Call log:
- waiting for locator('.editor-post-publish-panel__header-publish-button button.editor-post-publish-button__button')
- locator resolved to <button type="button" aria-disabled="false" class="components-button editor-post-publish-button editor-post-publish-button__button is-primary is-compact">Publish</button>
- attempting click action
2 × waiting for element to be visible, enabled and stable
- element is not stable
- retrying click action
- waiting 20ms
2 × waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
- <div class="components-modal__screen-overlay">…</div> intercepts pointer events
- retrying click action
- waiting 100ms
108 × waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
- <div class="components-modal__screen-overlay">…</div> intercepts pointer events
- retrying click action
- waiting 500ms
- waiting for element to be visible, enabled and stable
- element is not enabled
211 × retrying click action
- waiting 500ms
- waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
- <div class="components-modal__screen-overlay">…</div> intercepts pointer events
- retrying click action
- waiting 500ms
351 \|
352 \| await confirmPublishButton.waitFor( { state: 'visible', timeout: 10_000 } );
> 353 \| await confirmPublishButton.click();
\| ^
354 \|
355 \| // Wait for the "View Post" link to appear
356 \| await page
at /opt/ci/builds/builder/automattic/studio/e2e/site-navigation.test.ts:353:30
Error Context: test-results/site-navigation-Site-Navig-ddb91-e-permalink-structure-works/error-context.md
I'll iterate on handling these more robustly and ping you for another review soon.
Thanks for reviewing this a couple times @epeicher! I also tested on Mac and Windows using Parallels, and the suite is consistently passing for me as well. I think we're good to merge this and iterate on these tests if needed.
| Mac | Windows |
|---|---|
@epeicher @sejas could you please give this another test and approve this if it passes?