Failing openjdk17_hs_extended.openjdk_x86-64_alpine-linux jdk_tools tests (and Java 11 as well)

[Haroon-Khel]

The failed tests are

tools/jpackage/share/AppLauncherEnvTest.java.AppLauncherEnvTest tools/jpackage/share/jdk/jpackage/tests/JavaOptionsTest.java.JavaOptionsTest tools/jpackage/share/AddLauncherTest.java#id1.AddLauncherTest_id1 tools/jpackage/share/ArgumentsTest.java.ArgumentsTest tools/jpackage/share/EmptyFolderTest.java.EmptyFolderTest tools/jpackage/share/jdk/jpackage/tests/AppVersionTest.java.AppVersionTest tools/jpackage/share/jdk/jpackage/tests/BasicTest.java.BasicTest tools/jpackage/share/jdk/jpackage/tests/CookedRuntimeTest.java.CookedRuntimeTest tools/jpackage/share/jdk/jpackage/tests/DotInNameTest.java.DotInNameTest tools/jpackage/share/jdk/jpackage/tests/JLinkOptionsTest.java.JLinkOptionsTest tools/jpackage/share/jdk/jpackage/tests/JavaOptionsEqualsTest.java#id0.JavaOptionsEqualsTest_id0 tools/jpackage/share/jdk/jpackage/tests/JavaOptionsEqualsTest.java#id1.JavaOptionsEqualsTest_id1 tools/jpackage/share/jdk/jpackage/tests/MainClassTest.java.MainClassTest tools/jpackage/share/jdk/jpackage/tests/ModulePathTest.java.ModulePathTest tools/jpackage/share/jdk/jpackage/tests/ModulePathTest2.java.ModulePathTest2 tools/jpackage/share/jdk/jpackage/tests/ModulePathTest3.java.ModulePathTest3 tools/jpackage/share/jdk/jpackage/tests/MultipleJarAppTest.java.MultipleJarAppTest tools/jpackage/share/jdk/jpackage/tests/NoMPathRuntimeTest.java.NoMPathRuntimeTest

These failing tests are taking up 40G on our alpine docker hosts, often taking up all the space which causes disk space issues for our other tests

43.6G Test_openjdk17_hs_extended.openjdk_x86-64_alpine-linux_testList_0 43.8G Test_openjdk17_hs_extended.openjdk_x86-64_alpine-linux_testList_2

[sxa]

Re-running the whole extended suite without PARALLEL=dynamic so I can look at the space used and determine what we can skip to get most of the tests throgh: https://ci.adoptopenjdk.net/job/Test_openjdk11_hs_extended.openjdk_x86-64_alpine-linux/20/ (Running on on test-docker-alpine312-x64-2 which is hosted on the amd-1 host

[sxa]

There are two separate issues here - the disk space one which I am trying to track down, and the failure of some of the suites such as jdk_management_[01] and jdk_tools_[01] which will need to be separately diagnosed.

[Haroon-Khel]

Links to the job failures https://ci.adoptopenjdk.net/job/Test_openjdk17_hs_extended.openjdk_x86-64_alpine-linux_testList_0/28/console https://ci.adoptopenjdk.net/job/Test_openjdk17_hs_extended.openjdk_x86-64_alpine-linux_testList_2/16/console

[Haroon-Khel]

Seeing alot of unsatisfied link errors in the output

[2022-01-19T02:40:36.252Z] WARNING: Unknown module: me.mymodule.foo specified to --add-exports
[2022-01-19T02:40:36.252Z] Exception in thread "main" java.lang.UnsatisfiedLinkError: Can't load library: /home/jenkins/workspace/Test_openjdk17_hs_extended.openjdk_x86-64_alpine-linux_testList_0/aqa-tests/TKG/output_16425581511694/jdk_tools_0/work/scratch/2/test.31c02c75/output/JavaOptionsEqualsTest/lib/runtime/lib/libawt_xawt.so
[2022-01-19T02:40:36.252Z] 	at java.base/java.lang.ClassLoader.loadLibrary(Unknown Source)
[2022-01-19T02:40:36.252Z] 	at java.base/java.lang.Runtime.load0(Unknown Source)

I guess that's for this failing test tools/jpackage/share/jdk/jpackage/tests/JavaOptionsEqualsTest.java#id0.JavaOptionsEqualsTest_id0

[Haroon-Khel]

Found https://github.com/adoptium/aqa-tests/issues/2877 which accounts for the following tests

tools/jpackage/share/jdk/jpackage/tests/JavaOptionsTest.java.JavaOptionsTest
tools/jpackage/share/AddLauncherTest.java#id1.AddLauncherTest_id1
tools/jpackage/share/ArgumentsTest.java.ArgumentsTest
tools/jpackage/share/EmptyFolderTest.java.EmptyFolderTest
tools/jpackage/share/jdk/jpackage/tests/AppVersionTest.java.AppVersionTest
tools/jpackage/share/jdk/jpackage/tests/BasicTest.java.BasicTest
tools/jpackage/share/jdk/jpackage/tests/CookedRuntimeTest.java.CookedRuntimeTest
tools/jpackage/share/jdk/jpackage/tests/DotInNameTest.java.DotInNameTest
tools/jpackage/share/jdk/jpackage/tests/JLinkOptionsTest.java.JLinkOptionsTest
tools/jpackage/share/jdk/jpackage/tests/JavaOptionsEqualsTest.java#id0.JavaOptionsEqualsTest_id0
tools/jpackage/share/jdk/jpackage/tests/JavaOptionsEqualsTest.java#id1.JavaOptionsEqualsTest_id1
tools/jpackage/share/jdk/jpackage/tests/MainClassTest.java.MainClassTest
tools/jpackage/share/jdk/jpackage/tests/ModulePathTest.java.ModulePathTest
tools/jpackage/share/jdk/jpackage/tests/ModulePathTest2.java.ModulePathTest2
tools/jpackage/share/jdk/jpackage/tests/ModulePathTest3.java.ModulePathTest3
tools/jpackage/share/jdk/jpackage/tests/MultipleJarAppTest.java.MultipleJarAppTest
tools/jpackage/share/jdk/jpackage/tests/NoMPathRuntimeTest.java.NoMPathRuntimeTest

Still not accounted for tools/jpackage/share/AppLauncherEnvTest.java.AppLauncherEnvTest

[smlambert]

alpine-linux is a headless build (and we account for by this setting: https://github.com/adoptium/aqa-tests/blob/master/openjdk/openjdk.mk#L94-L100 so that the underlying jtreg framework will filter out tests that test a headful build).

Unsatisfied link errors unable to load a library that would not exist in a headless build libawt_xawt.so makes me think that the jtreg filtering is not working quite right (or as expected). Not sure if this is an upstream problem or we need some additional config settings in our makefile. Someone will need to check.

[sophia-guo]

Yes, those tests have been tagged as 'not workable for headless alpine linux' https://github.com/adoptium/aqa-tests/issues/2877.

Job output shows that option has been set up correctly.

21:36:29  "/home/jenkins/workspace/Test_openjdk17_hs_extended.openjdk_x86-64_alpine-linux_testList_0/openjdkbinary/j2sdk-image/bin/java" -Xmx512m -jar "/home/jenkins/workspace/Test_openjdk17_hs_extended.openjdk_x86-64_alpine-linux_testList_0/aqa-tests/TKG/../../jvmtest/openjdk/jtreg/lib/jtreg.jar" \
21:36:29  -agentvm -a -ea -esa -v:fail,error,time,nopass -retain:fail,error,*.dmp,javacore.*,heapdump.*,*.trc -ignore:quiet -timeoutFactor:8 -xml:verify -concurrency:24 -k:'!headful' -nativepath:"/home/jenkins/workspace/Test_openjdk17_hs_extended.openjdk_x86-64_alpine-linux_testList_0/openjdkbinary/openjdk-test-image/jdk/jtreg/native" -vmoptions:"-Xmx512m  -XX:+UseCompressedOops  -Djava.awt.headless=true" \

Similar tests belong to jdk_bean category has been filtered as expected in same build. I suspect those tests could be excluded by jtreg filter.

[sxa]

I

Seeing alot of unsatisfied link errors in the output

[2022-01-19T02:40:36.252Z] WARNING: Unknown module: me.mymodule.foo specified to --add-exports
[2022-01-19T02:40:36.252Z] Exception in thread "main" java.lang.UnsatisfiedLinkError: Can't load library: /home/jenkins/workspace/Test_openjdk17_hs_extended.openjdk_x86-64_alpine-linux_testList_0/aqa-tests/TKG/output_16425581511694/jdk_tools_0/work/scratch/2/test.31c02c75/output/JavaOptionsEqualsTest/lib/runtime/lib/libawt_xawt.so
[2022-01-19T02:40:36.252Z] 	at java.base/java.lang.ClassLoader.loadLibrary(Unknown Source)
[2022-01-19T02:40:36.252Z] 	at java.base/java.lang.Runtime.load0(Unknown Source)

I guess that's for this failing test tools/jpackage/share/jdk/jpackage/tests/JavaOptionsEqualsTest.java#id0.JavaOptionsEqualsTest_id0

I believe these are likely being caused as a result of the DISPLAY variable being set despite this being a headless build. We should look at inhibiting the explicit setting of that variable in the headless case.

[andrew-m-leonard]

Suspect we need a fix for Alpine here: https://github.com/adoptium/aqa-tests/blob/b3c533757347c4fdc6f9a01fc7cb503bb838dd70/buildenv/jenkins/JenkinsfileBase#L619

I'll try a patch

[smlambert]

auto exclude test jdk_tools plat=x86-64_alpine-linux

[Haroon-Khel]

@andrew-m-leonard You're right. I have just run two grinders, the first does not set DISPLAY (https://github.com/Haroon-Khel/openjdk-tests/blob/e5f12c17d75a6baece5bb4919cab5cedc021a823/buildenv/jenkins/JenkinsfileBase#L619) on alpine and the second does

https://ci.adoptopenjdk.net/job/Grinder/3525/console https://ci.adoptopenjdk.net/job/Grinder/3526/console

The second grinder shows unsatisfied link errors while the first does not. Either way the test fail both times and dumps are created both times

[Haroon-Khel]

I imagine there are some headless tests, which do require DISPLAY to be set, that do pass on alpine linux? If so, preventing DISPLAY from being set for all tests on alpine would not be an adequate solution. Unless I am mistaken

[andrew-m-leonard]

@Haroon-Khel So looks like the core is coming from:

13:31:19  pure virtual method called
13:31:19  terminate called without an active exception

Which is a native C/C++ issue, which would likely cause a core I guess

[sxa]

I imagine there are some headless tests, which do require DISPLAY to be set, that do pass on alpine linux? If so, preventing DISPLAY from being set for all tests on alpine would not be an adequate solution. Unless I am mistaken

DISPLAY is for a graphical X11 display. That will not typically exist on Alpine, so we should not be running any tests that require it, and if they do, they won't work due to the lissing libawt_xawt.so.

[sxa]

@andrew-m-leonard While it is happening in the same tests, the core issue (independent of the DISPLAY setting) should probably be tracked under https://github.com/adoptium/infrastructure/issues/2320

[sophia-guo]

Same for aarch64_alpine-linux