selenium icon indicating copy to clipboard operation
selenium copied to clipboard
verified publisher icon SeleniumHQ

[🐛 Bug]: "--headlessmode=new" with Chromedriver 128 in container results in SessionNotCreatedException

Open 1dEraNCeSIv0 opened this issue 1 year ago • 27 comments

What happened?

Recently we upgraded our Jenkins to the latest version and most of our (Java based) Selenium tests started failing in their pipelines, SessionNotCreatedException caused by a timeout in org.openqa.selenium.remote.http.AddSeleniumUserAgent.

Upon further investigation we found that the following combination of circumstances causes consistent failure:

  • Use chromium 128 / chromedriver 128
  • Use the new headless mode as --headless=new
  • Run from a docker container

I've browsed the issues here to check if it's been reported before and it looks similar to this issue, might be the same cause.

For now our workaround is to downgrade the chromium / chromedriver version our Jenkins runs with. We could also switch our tests to --headless=old but I see that as a fix of last resort. I'd much rather Selenium and new chromedriver versions work together out of the box, even in new headless mode.

How can we reproduce the issue?

See this repository for a minimal reproducing example. For instructions on how to reproduce the issue please see the readme.

Relevant log output

> Task :test

SeleniumTest > headlessNew() FAILED
    org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
    Host info: host: '83b1fb817f96', ip: '172.17.0.1'
        at app//org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:545)
        at app//org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:234)
        at app//org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:163)
        at app//org.openqa.selenium.chromium.ChromiumDriver.<init>(ChromiumDriver.java:114)
        at app//org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:88)
        at app//org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:83)
        at app//org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:72)
        at app//SeleniumTest.createChromeDriver(SeleniumTest.java:26)
        at app//SeleniumTest.headlessNew(SeleniumTest.java:9)

        Caused by:
        org.openqa.selenium.TimeoutException: java.util.concurrent.TimeoutException
        Build info: version: '4.23.0', revision: '4df0a231af'
        System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.0-18-amd64', java.version: '21.0.4'
        Driver info: driver.version: ChromeDriver
            at app//org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute0(JdkHttpClient.java:399)
            at app//org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
            at app//org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:55)
            at app//org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute(JdkHttpClient.java:355)
            at app//org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:89)
            at app//org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:75)
            at app//org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:61)
            at app//org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:162)
            at app//org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:216)
            at app//org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:174)
            at app//org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:527)
            ... 8 more

            Caused by:
            java.util.concurrent.TimeoutException
                at java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1960)
                at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2095)
                at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute0(JdkHttpClient.java:382)
                ... 18 more

SeleniumTest > headlessOld() STANDARD_ERROR
    Aug 29, 2024 12:25:03 PM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
    WARNING: Unable to find an exact match for CDP version 128, returning the closest version; found: 127; Please update to a Selenium version that supports CDP version 128

Gradle Test Executor 1 finished executing tests.

Operating System

Alpine 3.20, Debian 12

Selenium version

4.19.1, 4.23

What are the browser(s) and version(s) where you see this issue?

Chrome 128

What are the browser driver(s) and version(s) where you see this issue?

Chromedriver 128

Are you using Selenium Grid?

No

1dEraNCeSIv0 avatar Aug 29 '24 12:08 1dEraNCeSIv0

@1dEraNCeSIv0, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

github-actions[bot] avatar Aug 29 '24 12:08 github-actions[bot]

Thank you for sharing the details. I tried to reproduce the issue but was not able to.

Docker command: docker run --rm -it -p 4444:4444 -p 5900:5900 -p 7900:7900 --shm-size 2g selenium/standalone-chromium:latest

Selenium Java code:

public class ChromeHeadlessv128 {

  public static void main(String[] argv) throws Exception {
    ChromeOptions options = new ChromeOptions();
    options.addArguments("--headless=new");

    WebDriver driver = new RemoteWebDriver(options, false);
    driver.get("https://www.google.com/");

    driver.getTitle();
    driver.quit();
  }
}

Is the error happening each time? or is it intermittent? How can we reproduce this?

pujagani avatar Aug 30 '24 11:08 pujagani

I am able to reproduce it if I run multiple sessions in parallel or run multiple sessions sequentially.

pujagani avatar Aug 30 '24 12:08 pujagani

I tried reproducing it again locally and noticed that the image was broken due to line endings changing upon up- and download. I've also had to allow newer chrome versions as the alpine repo doesn't seem to keep the specific 128 version that was up to date yesterday available. Note that this means that once 129 becomes available the Dockerfile will probably build that into the image instead. But I don't know of any quick way to pin the version.

Long story short, I believe I've fixed the Dockerfile and the following steps should now work again to reproduce the issue using the repository linked above:

  • Clone the repo
  • Navigate to project root folder
  • run docker build -t reproducer . (or any other image name)
  • run docker run reproducer "/root/gradlew -i test"
  • wait for the timeout to happen (around 5min)

If there's any issues with the image please let me know. It should cause the issue consistently, my error-rate so far is 100% in maybe 10 attempts. Regarding parallelism or running multiple sessions, the demo repo above uses the default settings for all of these - but I'm not sure what these are.

1dEraNCeSIv0 avatar Aug 30 '24 12:08 1dEraNCeSIv0

@pujagani, can you try to reproduce the same again with image selenium/standalone-chromium:latest (updated on Aug-31 1:00 AM IST). I guess it appears from chromium version 128.0.6613.113

VietND96 avatar Sep 02 '24 18:09 VietND96

@VietND96 Thank you! Let me try it out and provide my findings here.

pujagani avatar Sep 09 '24 05:09 pujagani

I am able to reproduce the issue (using https://github.com/SeleniumHQ/selenium/issues/14457#issuecomment-2320919019)but not consistently, it failed one time with "selenium/standalone-chromium:latest" when using "options.addArguments("--headless=new");". Without headless or when using the old headless mode "options.addArguments("--headless");", it works as expected all the time though. Unable to find a pattern here.

pujagani avatar Sep 09 '24 05:09 pujagani

With the demo repo shared, I am able to see the error described in the issue. But those are two different things. I was trying to run tests on my machine pointing to the docker-selenium grid and was not able to reproduce the issue accurately on the last attempt. But the repo is trying to run tests inside the docker container locally without using the Grid. I have a feeling this is not a Selenium issue.

pujagani avatar Sep 09 '24 06:09 pujagani

In the demo repo shared I have made the following updates:

  1. Updated selenium to latest version :

testImplementation("org.seleniumhq.selenium:selenium-java:4.24.0")

  1. Updated the chromium and chromedriver versions in the dockerfile:

RUN apk add chromium>128.0.6613.119-r0 chromium-chromedriver>128.0.6613.119-r0

After this, I no longer see the error. Sharing the output below:

Caching disabled for task ':test' because:
  Build cache is disabled
Task ':test' is not up-to-date because:
  No history is available.
Starting process 'Gradle Test Executor 1'. Working directory: /root Command: /opt/java/openjdk/bin/java -Dorg.gradle.internal.worker.tmpdir=/root/build/tmp/test/work @/root/.gradle/.tmp/gradle-worker-classpath15735176167891063660txt -Xmx512m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

Gradle Test Executor 1 started executing tests.
Gradle Test Executor 1 finished executing tests.

> Task :test
Finished generating test XML results (0.005 secs) into: /root/build/test-results/test
Generating HTML test report...
Finished generating test html results (0.011 secs) into: /root/build/reports/tests/test

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.10/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 17s
2 actionable tasks: 1 executed, 1 up-to-date

pujagani avatar Sep 09 '24 06:09 pujagani

@1dEraNCeSIv0 Can you please try it and provide an update?

pujagani avatar Sep 09 '24 06:09 pujagani

I've incorporated your changes into the repository, no changes. Feel free to check if I made an error when editing the project.

Caching disabled for task ':test' because:
  Build cache is disabled
Task ':test' is not up-to-date because:
  No history is available.
Starting process 'Gradle Test Executor 1'. Working directory: /root Command: /opt/java/openjdk/bin/java -Dorg.gradle.internal.worker.tmpdir=/root/build/tmp/test/work @/root/.gradle/.tmp/gradle-worker-classpath6100839871221666423txt -Xmx512m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'

SeleniumTest > headlessNew() FAILED
    org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
    Host info: host: '42d9f1c3b18f', ip: '172.17.0.2'
        at app//org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:563)
        at app//org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:245)
        at app//org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:174)
        at app//org.openqa.selenium.chromium.ChromiumDriver.<init>(ChromiumDriver.java:114)
        at app//org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:88)
        at app//org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:83)
        at app//org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:72)
        at app//SeleniumTest.createChromeDriver(SeleniumTest.java:26)
        at app//SeleniumTest.headlessNew(SeleniumTest.java:9)

        Caused by:
        org.openqa.selenium.TimeoutException: java.util.concurrent.TimeoutException
        Build info: version: '4.24.0', revision: '748ffc9bc3'
        System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.153.1-microsoft-standard-WSL2', java.version: '21.0.4'
        Driver info: driver.version: ChromeDriver
            at app//org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute0(JdkHttpClient.java:418)
            at app//org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
            at app//org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:55)
            at app//org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute(JdkHttpClient.java:374)
            at app//org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:89)
            at app//org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:75)
            at app//org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:61)
            at app//org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:162)
            at app//org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:216)
            at app//org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:174)
            at app//org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:545)
            ... 8 more

            Caused by:
            java.util.concurrent.TimeoutException
                at java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1960)
                at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2095)
                at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute0(JdkHttpClient.java:401)
                ... 18 more

Gradle Test Executor 1 finished executing tests.

> Task :test FAILED

2 tests completed, 1 failed
Finished generating test XML results (0.01 secs) into: /root/build/test-results/test
Generating HTML test report...
Finished generating test html results (0.017 secs) into: /root/build/reports/tests/test

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///root/build/reports/tests/test/index.html

* Try:
> Run with --scan to get full insights.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
BUILD FAILED in 3m 29s


For more on this, please refer to https://docs.gradle.org/8.10/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
2 actionable tasks: 1 executed, 1 up-to-date

FWIW I also checked the specific installed chromium version when I run the image and it's the one you used

~ # apk list -i | grep chrome
chromium-chromedriver-128.0.6613.119-r0 x86_64 {chromium} (BSD-3-Clause) [installed]

1dEraNCeSIv0 avatar Sep 09 '24 10:09 1dEraNCeSIv0

Thank you for trying. I am not sure how to help further since I am unable to reproduce it consistently on my end.

pujagani avatar Sep 09 '24 11:09 pujagani

I've finally found that one of my colleagues can run the tests and they consistently work for them as well. I'll be digging more into that next week, hopefully I'll be able to narrow down the exact causes of the error. I'll let you know once I find out more

1dEraNCeSIv0 avatar Sep 12 '24 09:09 1dEraNCeSIv0

Okay, I've find some time to look into this further.

Turns out that my colleague who can run the tests consistently... cannot actually run them consistently. Upon retrying today they exhibited the same behavior I've mostly been getting.

So I've tried it on a windows machines and get the error 100% of the time I've tried it on a debian based linux machine and I get the error some 90% of the time My colleague tried on nixos and they got the same error My other colleague tried on a ubunut based linux distribution and they get it ... I don't know how often but they've at least seen success and error both

Now, ideally none of that would matter anyway because within the docker image we should all have the same state to run the same code with, getting the same results. Apparently this is not the case but at this point I have no idea what the differences are.

Some other things I tried: Upgrading to chromium / chromium-driver 129. It has had mixed results.

  • First I ran into a couple of instances where upgrading to 129 would apparently fix the issue for all further tests in the container. Up to 10 test executions in a row that would work.
  • Later I realized that the tests still fail once I run /root/gradlew clean after the update
  • I've also realized that even in headless=old mode the error can happen, it just happens more rarely

So the takeaway is that unspecified things happened that fixed the error temporarily but generally speaking 129 seems to exhibit the same issue we've been seeing with 128.

The only way I know that reliably avoids the timeout error is by using a chromium/chromedriver version < 127. And if it works perfectly on your end I wish what causes that.

1dEraNCeSIv0 avatar Sep 19 '24 15:09 1dEraNCeSIv0

I've made some changes and now I am on a setup that seems to work consistently so far. I've changed:

  • Away from Chromium to Google Chrome (downloaded as debian package from here)
  • Away from Chromium-Chromedriver to Google's Chromedriver (as fetched from the chromedriver for testing website).
  • Away from an alpine based image to an image based on ubuntu (was basically required for the first two changes)

With those changes made the selenium tests in the demo project now work again, seemingly consistently. I suspect the issue to be with chromium and / or chromium-chromedriver but that's all I've got.

In case anyone wants to take a look at what the working project looks like I've created a separate branch. ~~It'll eventually go out of sync but I'm too tired to fix it today~~ Edit: Maybe not a good solution but should stay in sync now.

1dEraNCeSIv0 avatar Sep 19 '24 18:09 1dEraNCeSIv0

I'm having the same issue... Upgrading to v128.0.6613.84 caused headless to break and return the same error as you had (session not created: DevToolsActivePort file doesn't exist). This only happens on our Jenkins CI, using headless=old works as a fix.. We're getting both Chrome and Chromedriver from their JSON endpoint (https://storage.googleapis.com/chrome-for-testing-public/...) but we're unsure of a solution that does not involve using the old headless.

LAguilera22 avatar Sep 20 '24 21:09 LAguilera22

On first glance this looks like a different issue, we've only ever gotten the error posted above and it does not mention DevToolsActivePort file doesn't exist

1dEraNCeSIv0 avatar Sep 21 '24 10:09 1dEraNCeSIv0

Unfortunately headless=old works, but only for some tests and in a non deterministic way. The only working solution is to downgrade to a version <= 127.

giacgbj avatar Sep 27 '24 06:09 giacgbj

I too was facing the same issue. Can see the following in chromedriver verbose logs

ERR: DisplayVkXcb.cpp:58 (initialize): xcb_connect() failed, error 1
[239:239:0927/103740.068107:ERROR:angle_platform_impl.cc(44)] Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
ERR: Display.cpp:1083 (initialize): ANGLE Display::initialize error 12289: Not initialized.
[239:239:0927/103740.068168:ERROR:gl_display.cc(497)] EGL Driver message (Critical) eglInitialize: Not initialized.
[239:239:0927/103740.068220:ERROR:gl_display.cc(767)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
[239:239:0927/103740.068282:ERROR:gl_display.cc(801)] Initialization of all EGL display types failed.
[239:239:0927/103740.068369:ERROR:gl_ozone_egl.cc(26)] GLDisplayEGL::Initialize failed.

Could be some issue with GPU Acceleration in latest chrome versions with headless mode.

Worked when i added --disable-gpu flag

KrishnaSuravarapu avatar Sep 27 '24 11:09 KrishnaSuravarapu

Worked when i added --disable-gpu flag

@KrishnaSuravarapu, I've always used that flag and your issue seems to be completely different from this one.

giacgbj avatar Sep 27 '24 11:09 giacgbj

Hi guys,

I faced this issue and after some tests with strace, it looks like chromium now tries to create a directory $HOME/.config/chromium/Crash Reports/ whereas the previous versions didn't (at least it didn't make it crash).

My test Dockerfile:

FROM debian:12-slim

RUN apt-get update -y && apt-get install -yq chromium-driver python3-pip strace
RUN pip3 install selenium==4.23.1 --break-system-packages

COPY test.py .

ENTRYPOINT ["python3", "test.py"]

test.py:

import re
import os
from selenium import webdriver

from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service

chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--headless')
chrome_options.add_argument('user-agent=test')

CHROMIUMDRIVER_PATH = os.environ.get('CHROMIUMDRIVER_PATH', "/usr/bin/chromedriver")

webdriver_service = Service(executable_path=CHROMIUMDRIVER_PATH)
driver = webdriver.Chrome(service=webdriver_service, options=chrome_options)

Test on read-only FS:

$ docker run -it --tmpfs "/tmp" --read-only plop bash
Traceback (most recent call last):
  File "//test.py", line 16, in <module>
    driver = webdriver.Chrome(service=webdriver_service, options=chrome_options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/chromium/webdriver.py", line 66, in __init__
    super().__init__(command_executor=executor, options=options)
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/remote/webdriver.py", line 212, in __init__
    self.start_session(capabilities)
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/remote/webdriver.py", line 299, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/remote/webdriver.py", line 354, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.11/dist-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited normally.
  (session not created: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x56082bf3eb9e <unknown>
#1 0x56082bc74020 <unknown>
#2 0x56082bcac561 <unknown>
#3 0x56082bca71cd <unknown>
#4 0x56082bcf6202 <unknown>
#5 0x56082bcf5823 <unknown>
#6 0x56082bce8cf3 <unknown>
#7 0x56082bcb3b08 <unknown>
#8 0x56082bcb4abe <unknown>
#9 0x56082bf10986 <unknown>
#10 0x56082bf13cb7 <unknown>
#11 0x56082bf13778 <unknown>
#12 0x56082bf14145 <unknown>
#13 0x56082bf0125e <unknown>
#14 0x56082bf1448f <unknown>
#15 0x56082beeb2d5 <unknown>
#16 0x56082bf2e205 <unknown>
#17 0x56082bf2e3e6 <unknown>
#18 0x56082bf3d9ba <unknown>
#19 0x7f2942a70144 <unknown>

With .config mounted:

$ docker run -it --tmpfs "/tmp" --tmpfs "/root/.config" --read-only plop bash

Just mounting a tmp FS on $HOME/.config does the trick. I hope it can help to move forward on this subject.

darkweaver87 avatar Oct 01 '24 10:10 darkweaver87

We're also running into this issue since Sep-18. Info from today's run:

Oct 02, 2024 1:25:38 PM org.openqa.selenium.remote.tracing.opentelemetry.OpenTelemetryTracer createTracer INFO: Using OpenTelemetry for tracing org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. Host info: host: 'ip-xx-xxx-x-xx.us-east-2.compute.internal', ip: 'xx.xxx.x.xx' at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:545) at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:234) at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:163) at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:143) at com.xxx.helper.DriverHelper.createRemoteDriver(DriverHelper.java:48)

Caused by: org.openqa.selenium.TimeoutException: java.util.concurrent.TimeoutException Build info: version: '4.23.0', revision: '4df0a231af' System info: os.name: 'Linux', os.arch: 'amd64', os.version: '4.14.291-218.527.amzn2.x86_64', java.version: '11.0.24' Driver info: driver.version: RemoteWebDriver at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute0(JdkHttpClient.java:399) at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42) at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:55) at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute(JdkHttpClient.java:355) at org.openqa.selenium.remote.tracing.TracedHttpClient.execute(TracedHttpClient.java:54) at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:89) at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:75) at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:61) at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:162) at org.openqa.selenium.remote.TracedCommandExecutor.execute(TracedCommandExecutor.java:53) at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:527) at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:234) at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:163) at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:143) at com.xxx.helper.DriverHelper.createRemoteDriver(DriverHelper.java:48)

We are trying to run using: Image: aws/codebuild/amazonlinux2-x86_64-standard:4.0 Environment type: Linux EC2 Compute type: EC2 Compute: 7 GB memory, 4 vCPUs

Please let me know if any more info is needed to dig deeper. All help appreciated.

qaquest avatar Oct 02 '24 22:10 qaquest

@qaquest Are you on Chromium or on Chrome? We've switched to Chrome and ever since this issue no longer happens. The main drawback is that Chrome is more annoying to set up properly but after way too much time spent we've since landed on something that works well for us.

Disclaimer: Allegedly setting up Chrome on some non-debian based distros is really unfun so you'd have to check if it works well with amazon linux.

1dEraNCeSIv0 avatar Oct 02 '24 22:10 1dEraNCeSIv0

@1dEraNCeSIv0 thx for the prompt reply! We use Chrome. Let me give some more context... We were on Selenium java 3.141.59. Running our regression using AWS CodeBuild. docker-compose had this info: `selenium-hub: image: selenium/hub:3.141.59-20200525 container_name: selenium-hub ports: - "4444:4444"

chrome: image: selenium/node-chrome:3.141.59-20200525 volumes: - /dev/shm:/dev/shm depends_on: - selenium-hub environment: - HUB_HOST=selenium-hub - HUB_PORT=4444`

The image used was “aws/codebuild/amazonlinux2-x86_64-standard:3.0”…this has been deprecated. Hence, we have to use “aws/codebuild/amazonlinux2-x86_64-standard:4.0". And with this change, the above referred error is showing up.

We then tried migrating to Selenium-4, using the following in docker-compose: `selenium-hub: image: selenium/hub:4.23.0 container_name: selenium-hub ports: - "4444:4444"

chrome: image: selenium/node-chrome:4.23.0 volumes: - /dev/shm:/dev/shm depends_on: - selenium-hub environment: - HUB_HOST=selenium-hub - HUB_PORT=4444`

Will try what you suggest & get back. In case you have any thoughts based on this additional info...thx!

qaquest avatar Oct 03 '24 01:10 qaquest

@bonigarcia referring to your comment, do you feel the above issue I mentioned could be similar? Working with “aws/codebuild/amazonlinux2-x86_64-standard:3.0” successfully…but deprecated. NOT working with “aws/codebuild/amazonlinux2-x86_64-standard:4.0"...but this is what is available currently with Amazon Linux.

qaquest avatar Oct 03 '24 21:10 qaquest

@qaquest I don't see the relationship of that comment with this issue, sorry.

bonigarcia avatar Oct 04 '24 07:10 bonigarcia

@qaquest I can confirm that the issue exists with google chrome when using the selenium/node-chrome image. I've added a branch to my reproducer repository that consistently reproduces the issue for me.

Okay I did some more testing. The issue seems to occur only with selenium/node-chrome:4.23.0 or higher. On selenium/node-chrome:4.22.0 I don't get the issue. I've checked the containers and it seems that 4.23.0 comes with Chrome 127 and 4.22.0 comes with Chrome 126. I've first seen these issues appear with Chromium 127, on Chromium 126 and lower everything works.

So it's my assumption is that Chromium made some more breaking changes in their 127 version that don't play well with Selenium, and the issue you're seeing is probably not because of the latest amazonlinux version itself but rather that you end up running Selenium with some 127+ version of Chrome and those versions are just cursed.

PS: If someone finds a site that publishes the changelogs for chrome / chromium version I'd be thankful. All I've found was a blog post that basically said "we're making everything better but btw we're no loger posting changelogs, too much work". 🤬

1dEraNCeSIv0 avatar Oct 04 '24 10:10 1dEraNCeSIv0

Hi, @1dEraNCeSIv0. This issue has been determined to require fixes in ChromeDriver.

You can see if the feature is passing in the Web Platform Tests.

If it is something new, please create an issue with the ChromeDriver team. Feel free to comment the issues that you raise back in this issue. Thank you.

github-actions[bot] avatar Nov 05 '24 12:11 github-actions[bot]

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.

github-actions[bot] avatar Dec 05 '24 22:12 github-actions[bot]