ionic-team
bug: e2e tests failing randomly
Prerequisites
- [x] I have read the Contributing Guidelines.
- [x] I agree to follow the Code of Conduct.
- [x] I have searched for existing issues that already report this problem, without success.
Stencil Version
4.26.0
Current Behavior
Having 500+ e2e tests and run them in parallel, you will see errors like that in your log:
● sdx-input-item › required checkbox › should have a space between label and asterisk when required field
Timeout - Async callback was not invoked within the 30000 ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 30000 ms timeout specified by jest.setTimeout.
at mapper (node_modules/jest-jasmine2/build/queueRunner.js:71:45)
● sdx-input-item › required checkbox › should have a space between label and asterisk when required field
TypeError: Cannot read properties of undefined (reading 'getProperty')
20 |
21 | it("should have a space between label and asterisk when required field", async () => {
> 22 | expect(await element.getProperty("required")).toBe(true)
| ^
23 | expect(element.innerText).toContain("Checkbox")
24 | expect(label.innerText).toBe(" *")
25 | })
at Object.<anonymous> (webcomponents/components/input-group/input-item/input-item.e2e.ts:22:28)
● sdx-input-item › checkbox without text › should be clickable
App did not load within 30000ms. Please ensure the content loads a stencil application.
9 |
10 | beforeEach(async () => {
> 11 | page = await newE2EPage({
| ^
12 | html: `
13 | <sdx-input-item type="checkbox" required>Checkbox</sdx-input-item>
14 | `,
at waitForStencil (node_modules/@stencil/core/testing/index.js:10557:11)
at e2eSetContent (node_modules/@stencil/core/testing/index.js:10550:3)
at newE2EPage (node_modules/@stencil/core/testing/index.js:10462:7)
at Object.<anonymous> (webcomponents/components/input-group/input-item/input-item.e2e.ts:11:14)
### Expected Behavior
no errors
### System Info
```shell
System: node 22.12.0
Platform: darwin (24.3.0)
CPU Model: Apple M1 Max (10 cpus)
Compiler: /node_modules/@stencil/core/compiler/stencil.js
Build: 1739293045
Stencil: 4.26.0 🐇
TypeScript: 5.5.4
Rollup: 2.56.3
Parse5: 7.1.2
jQuery: 4.0.0-pre
Terser: 5.31.1
Steps to Reproduce
No reproducer found. But there are at least 2 customers with the same errors. Some more info in another ticket from this comment downwards: https://github.com/ionic-team/stencil/issues/6072#issuecomment-2607531943
Code Reproduction URL
no sorry
Additional Information
Working workaround: add runInBand like stencil test --e2e --runInBand
@danyball, @christian-bromann Hi! Thanks for opening up the issue. I haven't tried this workaround with runInBand, because I didn't think it's a good solution. I also wasn't sure how to incorporate it into our pipeline, but either way, we are still waiting for a final proper fix for this issue. At the moment, we have removed the tests from github-actions as we were completely blocked by them, but I really hope the stencil team can come up with a solution for this issue.
Vi have also got this same issue, after upgrading from stencil v4.23 to v4.26 and puppeteer from v22 to v24.2.1. The issue only happens in our GitLab pipeline, and would cause 2-3 random e2e tests timeout when trying to set pageContent
Updating to @stencil/core: 4.27.2 from 4.18.3 - this issue is happening for me locally, not just in github actions. Using --runInBand workaround does prevent the error, but it makes the test suite take close to 5x longer to run (136.317 seconds versus 27.76 seconds for stencil test --e2e)
We are also experiencing some timeout issues, and we are using jest with React testing library for our tests. We have not tried to create a smaller reproduction, but we were working on upgrading to a newer version of Stencil along with some other changes. One test went from taking 75ms to complete to timing out at ~8s.
Same here! Using --runInBand is not a good workaround, it makes the tests too slow.
I'm still experiencing the issue even on Stencil 4.18.3.
What's the root cause of this issue?
@lucasoares We dont know, maybe you can find something? Maybe puppeteer has issues handling many many tests in parallel? But not sure where to start debugging that issue.
we have the same problem. i thought it was infrastructure problem.
here the log of stencil v4.22.3 🐤 and puppeteer 21:
$ stencil test --spec --e2e --coverage --silent
->
[58:27.3] @stencil/core
[58:27.9] v4.22.3 🐤
[58:28.9] testing e2e and spec files
[58:31.2] build, lib, dev mode, started ...
[58:31.5] transpile started ...
[58:35.1] transpile finished in 3.68 s
[58:36.0] generate custom elements + source maps started ...
[58:36.0] generate lazy + source maps started ...
[58:47.9] generate custom elements + source maps finished in 11.94 s
[58:48.6] generate lazy + source maps finished in 12.60 s
[58:48.6] copy started ...
[58:49.0] copy finished (525 files) in 357 ms
[58:49.1] build finished in 17.89 s
[ ERROR ] Timed out after waiting 30000ms TimeoutError: Timed out after
waiting 30000ms at
/var/lib/jenkins/workspace/node_modules/puppeteer-core/lib/cjs/puppeteer/common/util.js:287:19
at
/var/lib/jenkins/workspace/node_modules/puppeteer-core/lib/cjs/third_party/rxjs/rxjs.js:3:59978
at f._next
(/var/lib/jenkins/workspace/node_modules/puppeteer-core/lib/cjs/third_party/rxjs/rxjs.js:3:11821)
at t.next
(/var/lib/jenkins/workspace/node_modules/puppeteer-core/lib/cjs/third_party/rxjs/rxjs.js:3:6459)
at t.<anonymous>
(/var/lib/jenkins/workspace/node_modules/puppeteer-core/lib/cjs/third_party/rxjs/rxjs.js:3:71381)
at t._execute
(/var/lib/jenkins/workspace/node_modules/puppeteer-core/lib/cjs/third_party/rxjs/rxjs.js:3:28234)
at t.execute
(/var/lib/jenkins/workspace/node_modules/puppeteer-core/lib/cjs/third_party/rxjs/rxjs.js:3:28057)
at t.flush
(/var/lib/jenkins/workspace/node_modules/puppeteer-core/lib/cjs/third_party/rxjs/rxjs.js:3:32540)
at listOnTimeout (node:internal/timers:588:17) at
process.processTimers (node:internal/timers:523:7)
i solved this for me with stencil v4.22.3 + puppeteer-core v24.20.0 + Chrome v139:
// stencil.config.ts
testing: {
browserArgs: ['--disable-gpu'],
}
Found via: https://pptr.dev/troubleshooting#running-on-alpine -> Github Issue https://github.com/puppeteer/puppeteer/issues/11640#issuecomment-2416848786
My problem was a newer chrome version v139 with older puppeteer-core version v21.x and the missing argument which force disables the gpu, See puppeteer supported browserlist https://pptr.dev/supported-browsers
Cheers