[Feature] Add expect.fail() with same idea as test.fail() to specify place where failure is expected

[AndriiFrolov]

I actively use test.fail option when some bugs prevents test from passing. Reasons why I do this:

Let's say you have 30% tests failing due to known bug. And you analyse failures each day or even after each dev code pull request.

You don't want to skip test, because difficult to track condition when it should be enabled back. Also you can forget about that (human factor). Also failure may be due to minor bug that does not prevent further execution. expect.soft() does not solve all problems cause it will still mark test as failed

So test.fail is super feature that allows to run tests and get notified when it actually passed instead of fail.

However, It does not allow to specify which exactly step expects to be failing. Let's say you have just minor issue that does not prevent you from completing main test flow. When you mark the whole test as test.fail, you don't get notified if smth is wrong AFTER your know bug (which again may be very minor)

What I would like to see is the following:

expect.fail()
expect.soft.fail()

AND/OR possibility to mark block of code (perhaps this is easier to implement):

await test.fail(`This block of code is expect to fail due to bug 12345`, async () => {
      await expect.soft(row.getCell('# NCR')).not.toContainText('Draft');
      await expect.soft(row.getCell('Stage')).toHaveText('Pending Assignment');
      await expect.soft(row.getCell('Assigned To')).toHaveText('-');
    });

[AndriiFrolov]

@pavelfeldman seems this functionality is supported by other users, any plans to consider it? I think having test.fail() as code block like test.step() is ideal solution

[AndriiFrolov]

@mxschmitt @pavelfeldman any chances to get this in backlog?

[VasilySazonenko]

I actively use test.fail option when some bugs prevents test from passing. Reasons why I do this:

Let's say you have 30% tests failing due to known bug. And you analyse failures each day or even after each dev code pull request.

You don't want to skip test, because difficult to track condition when it should be enabled back. Also you can forget about that (human factor). Also failure may be due to minor bug that does not prevent further execution. expect.soft() does not solve all problems cause it will still mark test as failed

So test.fail is super feature that allows to run tests and get notified when it actually passed instead of fail.

However, It does not allow to specify which exactly step expects to be failing. Let's say you have just minor issue that does not prevent you from completing main test flow. When you mark the whole test as test.fail, you don't get notified if smth is wrong AFTER your know bug (which again may be very minor)

What I would like to see is the following:

expect.fail()
expect.soft.fail()

AND/OR possibility to mark block of code (perhaps this is easier to implement):

await test.fail(`This block of code is expect to fail due to bug 12345`, async () => {
      await expect.soft(row.getCell('# NCR')).not.toContainText('Draft');
      await expect.soft(row.getCell('Stage')).toHaveText('Pending Assignment');
      await expect.soft(row.getCell('Assigned To')).toHaveText('-');
    });

you can use expect(false, "Your message").toBeTruthy() or if you do not want to stop execution you can use expect.soft(false, "Your message").toBeTruthy()

[AndriiFrolov]

I actively use test.fail option when some bugs prevents test from passing. Reasons why I do this:

Let's say you have 30% tests failing due to known bug. And you analyse failures each day or even after each dev code pull request.

You don't want to skip test, because difficult to track condition when it should be enabled back. Also you can forget about that (human factor). Also failure may be due to minor bug that does not prevent further execution. expect.soft() does not solve all problems cause it will still mark test as failed

So test.fail is super feature that allows to run tests and get notified when it actually passed instead of fail.

However, It does not allow to specify which exactly step expects to be failing. Let's say you have just minor issue that does not prevent you from completing main test flow. When you mark the whole test as test.fail, you don't get notified if smth is wrong AFTER your know bug (which again may be very minor)

What I would like to see is the following:

expect.fail()

expect.soft.fail()

AND/OR possibility to mark block of code (perhaps this is easier to implement):

await test.fail(This block of code is expect to fail due to bug 12345, async () => {

  await expect.soft(row.getCell('# NCR')).not.toContainText('Draft');

  await expect.soft(row.getCell('Stage')).toHaveText('Pending Assignment');

  await expect.soft(row.getCell('Assigned To')).toHaveText('-');

});

you can use

expect(false, "Your message").toBeTruthy()

or if you do not want to stop execution you can use

expect.soft(false, "Your message").toBeTruthy()

That is very far from what I described