cli icon indicating copy to clipboard operation
cli copied to clipboard
verified publisher icon forcedotcom

apex:run:test does not return until minutes specified in --wait parameter have elapsed

Open gsbasso opened this issue 4 months ago • 7 comments

Summary

Have noticed the test stage in our Jenkins builds always complete in 90 minutes, which is the exact value specified in the --wait parameter. It's hard to believe it takes exactly the same amount of time for every build.

Over time we have gradually increased the --wait value as we add more tests to avoid timeouts.

Steps To Reproduce

  1. Run the command sf apex:run:test --result-format junit --code-coverage --synchronous --wait 90
  2. Note how long it takes for command to return/complete.

Expected result

Expect test command to return/complete once all tests have completed.

Actual result

Command appears to return/complete after the value specified in the --wait parameter

Additional information

If the actual result turns out to be the expected result what is the suggestion for executing tests are part of a CI/CD process?

System Information

{
  "architecture": "win32-x64",
  "cliVersion": "@salesforce/cli/2.104.6",
  "nodeVersion": "node-v20.12.2",
  "osVersion": "Windows_NT 10.0.22631",
  "rootPath": "C:\\Users\\XXXXXXX\\AppData\\Roaming\\npm\\node_modules\\@salesforce\\cli",
  "shell": "powershell",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 3.2.34 (core)",
    "@oclif/plugin-commands 4.1.32 (core)",
    "@oclif/plugin-help 6.2.32 (core)",
    "@oclif/plugin-not-found 3.2.66 (core)",
    "@oclif/plugin-plugins 5.4.46 (core)",
    "@oclif/plugin-search 1.2.28 (core)",
    "@oclif/plugin-update 4.7.4 (core)",
    "@oclif/plugin-version 2.2.32 (core)",
    "@oclif/plugin-warn-if-update-available 3.1.46 (core)",
    "@oclif/plugin-which 3.2.39 (core)",
    "@salesforce/cli 2.104.6 (core)",
    "agent 1.24.6 (core)",
    "apex 3.6.19 (core)",
    "api 1.3.3 (core)",
    "auth 3.9.0 (core)",
    "community 3.3.33 (link) C:\\Users\\XXXXXXX\\AppData\\Roaming\\npm\\node_modules\\@salesforce\\plugin-community",
    "data 4.0.55 (core)",
    "deploy-retrieve 3.23.0 (core)",
    "info 3.4.83 (core)",
    "limits 3.3.65 (core)",
    "marketplace 1.3.8 (core)",
    "org 5.9.25 (core)",
    "packaging 2.19.7 (core)",
    "schema 3.3.79 (core)",
    "settings 2.4.44 (core)",
    "signups 2.3.12 (link) C:\\Users\\XXXXXXX\\AppData\\Roaming\\npm\\node_modules\\@salesforce\\plugin-signups",
    "sobject 1.4.69 (core)",
    "telemetry 3.6.55 (core)",
    "templates 56.3.61 (core)",
    "trust 3.7.113 (core)",
    "user 3.6.35 (core)"
  ]
}

gsbasso avatar Sep 11 '25 18:09 gsbasso

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

github-actions[bot] avatar Sep 11 '25 18:09 github-actions[bot]

Hello @gsbasso :wave: None of the versions of sf you shared match the latest release.

Shared: 2.104.6 Latest: 2.105.6

Update to the latest version of Salesforce CLI (docs) and confirm that you're still seeing your issue. You can also try the rc and nightly releases! (docs)

After updating, share the full output of sf version --verbose --json

github-actions[bot] avatar Sep 11 '25 18:09 github-actions[bot]

Updating to 2.105.6 made no difference

gsbasso avatar Sep 16 '25 19:09 gsbasso 

{
  "architecture": "win32-x64",
  "cliVersion": "@salesforce/cli/2.105.6",
  "nodeVersion": "node-v22.19.0",
  "osVersion": "Windows_NT 10.0.17763",
  "rootPath": "D:\\Program Files\\sf\\client",
  "shell": "cmd.exe",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 3.2.34 (core)",
    "@oclif/plugin-commands 4.1.32 (core)",
    "@oclif/plugin-help 6.2.32 (core)",
    "@oclif/plugin-not-found 3.2.67 (core)",
    "@oclif/plugin-plugins 5.4.46 (core)",
    "@oclif/plugin-search 1.2.28 (core)",
    "@oclif/plugin-update 4.7.4 (core)",
    "@oclif/plugin-version 2.2.32 (core)",
    "@oclif/plugin-warn-if-update-available 3.1.46 (core)",
    "@oclif/plugin-which 3.2.39 (core)",
    "@salesforce/cli 2.105.6 (core)",
    "agent 1.24.7 (core)",
    "apex 3.6.19 (core)",
    "api 1.3.3 (core)",
    "auth 3.9.1 (core)",
    "community 3.2.19 (link) D:\\Program Files\\sf\\client\\node_modules\\@salesforce\\plugin-community",
    "data 4.0.55 (core)",
    "deploy-retrieve 3.23.0 (core)",
    "info 3.4.84 (core)",
    "limits 3.3.66 (core)",
    "marketplace 1.3.8 (core)",
    "org 5.9.27 (core)",
    "packaging 2.19.8 (core)",
    "schema 3.3.80 (core)",
    "settings 2.4.45 (core)",
    "sobject 1.4.70 (core)",
    "telemetry 3.6.55 (core)",
    "templates 56.3.63 (core)",
    "trust 3.7.113 (core)",
    "user 3.6.36 (core)"
  ]
}

gsbasso avatar Sep 16 '25 19:09 gsbasso

Hey @gsbasso, I have a few questions for you:

  • When it hits the timeout (at 90m), does it show the full results? or does it show a testRunId for you to use with the sf apex get test command?
  • Does this only happen in Jenkins?
    • Can you reproduce this on your local machine? I am unable to on a Mac.
  • Did this just recently become an issue? If so, do you know what version of the cli started causing this issue?

iowillhoit avatar Oct 02 '25 21:10 iowillhoit

Hi @iowillhoit

Apologies for the delayed response...

Answer to first question is it shows full results.

Answer to second is I will have to get back to you as I rarely run all tests on my local machine (only when I need to update coverage results on SonarQube). Plus I will have to think how I'm going to get start/end times.

I only noticed this recently, and at that time the CLI version on our Jenkins server was 2.75.5 from back in Feb 2025. So it's at least as far back as that. We have since recently updated CLI version on Jenkins server but that made no difference.

gsbasso avatar Oct 06 '25 20:10 gsbasso

Not sure if anyone is looking into this but here is an extract from a (relatively) recent test-results.xml file.

testTotalTime amounts to 85 minutes. But based on the start time, 3:37pm, and the last modified time on the file itself, which is 5:19pm, the entire process took 102 minutes.

I'm not sure what conclusions can be drawn, if any.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
    <testsuite name="force.apex" timestamp="2025-10-07T19:37:18.000Z" hostname="https://enterprise-speed-2198.scratch.my.salesforce.com" tests="947" failures="1"  errors="0"  time="5049.94">
        <properties>
            <property name="outcome" value="Failed"/>
            <property name="testsRan" value="947"/>
            <property name="passing" value="946"/>
            <property name="failing" value="1"/>
            <property name="skipped" value="0"/>
            <property name="passRate" value="100%"/>
            <property name="failRate" value="0%"/>
            <property name="testStartTime" value="Tue Oct 07 2025 3:37:18 PM"/>
            <property name="testSetupTimeInMs" value="74840"/>
            <property name="testExecutionTime" value="5049.94 s"/>
            <property name="testTotalTime" value="5124.78 s"/>
            <property name="commandTime" value="0.22 s"/>
            <property name="hostname" value="https://enterprise-speed-2198.scratch.my.salesforce.com"/>
            <property name="orgId" value="00DKU0000001rvj2AA"/>
            <property name="username" value="[email protected]"/>
            <property name="testRunId" value="707KU00000Fmccj"/>
            <property name="userId" value="005KU000000LNEkYAO"/>
            <property name="coveredLines" value="15225"/>
            <property name="testRunCoverage" value="89%"/>
            <property name="orgWideCoverage" value="88%"/>
        </properties>

gsbasso avatar Nov 06 '25 19:11 gsbasso