playwright icon indicating copy to clipboard operation
playwright copied to clipboard

[Feature] Have critical section support for tests so projects don't run them in parallel

Open sumanthpfs opened this issue 3 years ago • 5 comments

can I limit workers to 1 on a single test file? - so as to not run the test suite sequentially I have my default projects set to 3 browsers to support multi-browser tests on a single run. I have a test case that has access to a resource that cannot be shared across the 3 browsers simultaneously. In order to avoid this race condition, I had to run this test on a single worker - so all three browsers don't run in parallel. I tried making the test file serial by test.describe.configure({ mode: 'serial' }); and test.describe.serial(title, callback) but none of them had any effect on running the test serially across the 3 browsers. I assume the playwright doesn't support serializing the test on a project level.

`projects: [ { name: 'chromium', use: { ...devices['Desktop Chrome'], }, },

{
  name: 'firefox',
  use: {
    ...devices['Desktop Firefox'],
  },
},

{
  name: 'webkit',
  use: {
    ...devices['Desktop Safari'],
  },
},`

sumanthpfs avatar May 11 '22 00:05 sumanthpfs

Please, see link below, maybe it could help.

https://playwright.dev/docs/test-parallel#disable-parallelism

ltsuda avatar May 11 '22 03:05 ltsuda

If you have multiple projects and the worker count is not 1, they will always run in parallel. Usually we recommend designing tests that they can run in parallel and are not depended on each other.

mxschmitt avatar May 11 '22 10:05 mxschmitt

Th

Please, see link below, maybe it could help.

https://playwright.dev/docs/test-parallel#disable-parallelism

I observe two features that come close to solving this issue. The serial mode and limiting workers. Serial Mode: test.describe.configure({ mode: 'serial' });` This cmd serializes a single test file making sure all the test cases in the file run in order. But it doesn't serialize the way WebKit, firefox, and chromium run their tests - Creating a race condition between webkit, firefox, and chromium.

Limit Workers: npx playwright test --workers 1 setting my worker to 'one' helps me not get into a race condition between WebKit, firefox, and chromium but makes all the tests in the suite run serially. This approach wouldn't make me take advantage of parallelization among all the tests in the suite.

sumanthpfs avatar May 12 '22 21:05 sumanthpfs

If you have multiple projects and the worker count is not 1, they will always run in parallel. Usually we recommend designing tests that they can run in parallel and are not depended on each other.

The test case is independent of other test cases in the test file. The race condition happens to come on its own. As the same test case is being run on WebKit, Firefox, and Chromium in parallel causing the issue.

Support to control the order of WebKit, Firefox, and chromium on a single test file would be a helpful feature.

sumanthpfs avatar May 12 '22 21:05 sumanthpfs

May be solved by https://github.com/microsoft/playwright/issues/14895#issuecomment-1219986634

rwoll avatar Aug 19 '22 23:08 rwoll

Why was this issue closed?

Thank you for your involvement. This issue was closed due to limited engagement (upvotes/activity), lack of recent activity, and insufficient actionability. To maintain a manageable database, we prioritize issues based on these factors.

If you disagree with this closure, please open a new issue and reference this one. More support or clarity on its necessity may prompt a review. Your understanding and cooperation are appreciated.

pavelfeldman avatar Nov 16 '23 04:11 pavelfeldman