Plugin "retryFailedStep" doesn't work in --debug or --verbose mode, searching for element fails immediately

[mirao]

It's a regression in 3.6.0. It works well with 3.5.15.

What are you trying to achieve?

You should see the Retrying... Attempt ... retries in the following test.

This is how it works with 3.5.15 :heavy_check_mark:

CodeceptJS v3.5.15 #StandWithUkraine
Using test root "/home/mirao/workspace/codeceptjs/tests/my"
Helpers: Playwright
Plugins: screenshotOnFail, retryFailedStep

My --
    [1]  Starting recording promises
    Timeouts: 
 › [Session] Starting singleton browser session
  test something
    I see element ".blabla"
    [1] Retrying... Attempt #2
    [1] Retrying... Attempt #3
    [1] Retrying... Attempt #4
    [1] Error (Non-Terminated) | Error: Element ".blabla" is not visible on page. | (err) => { step.status = 'failed'; step.endTime = ...
    [1] Error | Error: Element ".blabla" is not visible on page. undefined...
    [1] <teardown> Stopping recording promises
 › <screenshotOnFail> Test failed, try to save a screenshot
 › Screenshot is saving to /home/mirao/workspace/codeceptjs/tests/my/output/test_something.failed.png
  ✖ FAILED in 1148ms

    [2]  Starting recording promises

-- FAILURES:

  1) My
       test something:
     Element ".blabla" is not visible on page.
      at dontSeeElementError (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/codeceptjs/lib/helper/errors/ElementAssertion.js:23:9)
      at Playwright.seeElement (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/codeceptjs/lib/helper/Playwright.js:1460:7)
  
  Scenario Steps:
  - I.seeElement(".blabla") at Test.<anonymous> (./My_test.ts:4:7)
  
  Artifacts:
  - screenshot: /home/mirao/workspace/codeceptjs/tests/my/output/test_something.failed.png


  FAIL  | 0 passed, 1 failed   // 2s

What do you get instead?

Provide console output if related. Use --verbose mode for more details.

This is how it works with 3.6.0 or 3.6.2 :bug:

CodeceptJS v3.6.0 #StandWithUkraine
Using test root "/home/mirao/workspace/codeceptjs/tests/my"
Helpers: Playwright
Plugins: screenshotOnFail, retryFailedStep

My --
    [1]  Starting recording promises
    Timeouts: 
 › [Session] Starting singleton browser session
  test something
    I see element ".blabla"
    [1]  Error (Non-Terminated) | Error: Element ".blabla" is not visible on page. | (err) => { step.status = 'failed'; step.endTime = ...
    [1] Error | Error: Element ".blabla" is not visible on page. undefined...
    [1] <teardown>  Stopping recording promises
 › <screenshotOnFail> Test failed, try to save a screenshot
 › Screenshot is saving to /home/mirao/workspace/codeceptjs/tests/my/output/test_something.failed.png
  ✖ FAILED in 117ms

    [2]  Starting recording promises

-- FAILURES:

  1) My
       test something:
     Element ".blabla" is not visible on page.
      at dontSeeElementError (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/codeceptjs/lib/helper/errors/ElementAssertion.js:23:9)
      at Playwright.seeElement (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected][email protected][email protected]/node_modules/codeceptjs/lib/helper/Playwright.js:1464:7)
  
  Scenario Steps:
  - I.seeElement(".blabla") at Test.<anonymous> (./My_test.ts:4:7)
  
  Artifacts:
  - screenshot: /home/mirao/workspace/codeceptjs/tests/my/output/test_something.failed.png


  FAIL  | 0 passed, 1 failed   // 574ms

Provide test source code if related

Feature("My");

Scenario("test something", ({ I }) => {
    I.seeElement(".blabla");
});

Details

• CodeceptJS version: 3.6.0 or 3.6.2
• NodeJS Version: 18.20
• Operating System: Ubuntu 22.04
• Playwright Version 1.43.1
• Configuration file:

export const config: CodeceptJS.MainConfig = {
    tests: "./*_test.ts",
    output: "./output",
    helpers: {
        Playwright: {
            browser: "chromium",
            url: "http://localhost",
            show: true,
        },
    },
    plugins: {
        retryFailedStep: {
            enabled: true,
        },
    },
    name: "my",
};

[mirao]

Interestingly it works well in a "shell" mode (tested with 3.6.3):

$ codeceptjs shell --verbose
Interactive shell started
 Use JavaScript syntax to try steps in action
 - Press ENTER to run the next step
 - Press TAB twice to see all available commands
 - Type exit + Enter to exit the interactive shell
 - Prefix => to run js commands 
 I.seeElement(".blabla")
    [1] <pause> Retrying... Attempt #2
    [1] <pause> Retrying... Attempt #3

[mirao]

Well, only --debug or --verbose mode is affected. And if I comment out this line: https://github.com/codeceptjs/CodeceptJS/blob/e34471c85bc1758ab42c5db30b16d17bdbda3b85/lib/command/run.js#L17 everything works well even in --verbose mode.

store.debugMode seems to be needed for healers only? https://github.com/codeceptjs/CodeceptJS/pull/4230

@DavertMik

[DavertMik]

Thanks, I'm checking

[mirao]

I "solved" it a different way to myself. I patched the retryFailedStep plugin - I removed this line https://github.com/codeceptjs/CodeceptJS/blob/ac75d948533609523057be3a52539a25edf1c2f1/lib/plugin/retryFailedStep.js#L96 It seems to me that the removing doesn't affect functionality of the plugin. I tested the plugin in non-verbose mode, --verbose mode, in shell mode and for pause()

[github-actions[bot]]

This issue is stale because it has been open for 90 days with no activity.

[javierfuentesm]

Are there any updates? @DavertMik 🫣 im facing the same issue

[emiliangrecu]

I am also facing this problem.