HDFS-16740. Mini cluster test flakiness

[snmvaughan]

Description of PR

Mini clusters used during HDFS unit tests are reporting test failures that do not appear to be directly related to submitted changes. The failures are the result of either interactions between tests run in parallel, or tests which share common disk space for tests. In all cases, the tests can be run individually serially without any errors. Addressing this issue will simplify future submissions by eliminating the confusion introduced by these unrelated test failures.

We can apply lessons recently from TestRollingUpgrade, which was recently patched to unblock a recent submission. The fixes involved changing the HDFS configuration to use temporary disk space for each individual tests, and using try-with-resources to ensure that clusters were shutdown cleanly.

Add a constructor to the mini cluster constructors and builders that takes a TemporaryFolder. This encourages the use of managed temporary folders that are unique to the test and automatically cleaned up. The field is named "baseDir" to communicate that this folder acts as the base directory for test data. Using the new constructor encourages the consistent use of the managed temporary folder.

Allow the same baseDir to be provided multiple times, simplifying the coding of cluster restarts. This simplifies these tests as the use of the same TemporaryFolder will always work (even for restarts)

This change also reduces the number of tests that make assumptions about the mini clusters using encapsulation. These changes avoid configurations that directly use HDFS_MINIDFS_BASEDIR.

How was this patch tested?

Both Maven Surefire and an IDE were used to demonstrate that the tests operate as expected.

For code changes:

• [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• [ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[snmvaughan]

@steveloughran You mentioned that the test system could use some attention. The set of changes in this pull request, using an updated MiniDFSCluster that leverages temporary folders and ensuring that the cluster is shutdown at the end of the test, could be applied to the remaining tests.

[hadoop-yetus]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 54s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 1s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	test4tests	0m 0s		The patch appears to include 13 new or modified test files.
			_ trunk Compile Tests _
+1 :green_heart:	mvninstall	39m 23s		trunk passed
+1 :green_heart:	compile	1m 32s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	compile	1m 29s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart:	checkstyle	1m 19s		trunk passed
+1 :green_heart:	mvnsite	1m 39s		trunk passed
+1 :green_heart:	javadoc	1m 25s		trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	javadoc	1m 46s		trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart:	spotbugs	3m 32s		trunk passed
+1 :green_heart:	shadedclient	23m 4s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	1m 23s		the patch passed
+1 :green_heart:	compile	1m 23s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	javac	1m 23s		the patch passed
+1 :green_heart:	compile	1m 18s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart:	javac	1m 18s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	1m 0s		hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 257 unchanged - 1 fixed = 257 total (was 258)
+1 :green_heart:	mvnsite	1m 27s		the patch passed
+1 :green_heart:	javadoc	0m 58s		the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 :green_heart:	javadoc	1m 25s		the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 :green_heart:	spotbugs	3m 23s		the patch passed
+1 :green_heart:	shadedclient	22m 45s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 :green_heart:	unit	248m 31s		hadoop-hdfs in the patch passed.
+1 :green_heart:	asflicense	0m 57s		The patch does not generate ASF License warnings.
		358m 37s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/4/artifact/out/Dockerfile
GITHUB PR	https://github.com/apache/hadoop/pull/4835
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux cb0727a11da3 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / a5ed6afa9d603ca30c83ab8ed3cc49cf0972d045
Default Java	Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/4/testReport/
Max. process+thread count	3513 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[GuoPhilipse]

indeed, restartNameNode faces timeout from time to time, for now we have #5377. it will be very nice if we can eliminate this test case errors . do we need to change all places related new MiniDFSCluster.Builder to the new constructor ?

[snmvaughan]

From what I've seen, many of the timeouts are related to:

• Resources that aren't shutdown appropriately, leaving background threads that aren't daemon threads
• Interactions between tests when run concurrently, where the tests operate using the same space

This fix is focussed on both issues, as it uses a TemporaryFolder to avoid sharing space between tests and ensures that the cluster and its resources are always shutdown. I'd hope that we would expand this to the remaining uses of the mini clusters. Originally I had proposed new calls that actually took a TemporaryFolder as a parameter, thinking we'd eventually phase out the File-based calls and drop them entirely to ensure consistency.

In addition, I've been submitting PRs that address some of the threading issues (e.g. HDFS-16904 and HADOOP-18279.

[GuoPhilipse]

Thanks @snmvaughan for you nice idea, comments from @xinglin seems make sense ,Could you review and resubmit new changes ?

[hadoop-yetus]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	1m 29s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 1s		The patch does not contain any @author tags.
+1 :green_heart:	test4tests	0m 0s		The patch appears to include 13 new or modified test files.
			_ trunk Compile Tests _
+1 :green_heart:	mvninstall	46m 54s		trunk passed
+1 :green_heart:	compile	1m 35s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 :green_heart:	compile	1m 27s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 :green_heart:	checkstyle	1m 9s		trunk passed
+1 :green_heart:	mvnsite	1m 39s		trunk passed
+1 :green_heart:	javadoc	1m 11s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 :green_heart:	javadoc	1m 37s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 :green_heart:	spotbugs	3m 44s		trunk passed
+1 :green_heart:	shadedclient	28m 26s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	1m 34s		the patch passed
+1 :green_heart:	compile	1m 34s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 :green_heart:	javac	1m 34s		the patch passed
+1 :green_heart:	compile	1m 24s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 :green_heart:	javac	1m 24s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 58s		hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 257 unchanged - 1 fixed = 257 total (was 258)
+1 :green_heart:	mvnsite	1m 31s		the patch passed
+1 :green_heart:	javadoc	0m 57s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 :green_heart:	javadoc	1m 42s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 :green_heart:	spotbugs	3m 50s		the patch passed
+1 :green_heart:	shadedclient	30m 5s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 :x:	unit	304m 33s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
-1 :x:	asflicense	0m 46s	/results-asflicense.txt	The patch generated 2 ASF License warnings.
		435m 23s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestFileCorruption
	hadoop.hdfs.TestBlocksScheduledCounter
	hadoop.hdfs.TestDataTransferProtocol
	hadoop.hdfs.TestSetrepIncreasing
	hadoop.hdfs.TestReadStripedFileWithDecodingDeletedData
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy
	hadoop.hdfs.server.namenode.ha.TestObserverNode
	hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor
	hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
	hadoop.hdfs.TestDFSOutputStream
	hadoop.hdfs.TestDistributedFileSystemWithECFile

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/5/artifact/out/Dockerfile
GITHUB PR	https://github.com/apache/hadoop/pull/4835
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 6ff65efba137 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 20729390c455eee5882790a777f7d8522e50c0a7
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/5/testReport/
Max. process+thread count	2805 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[snmvaughan]

There are 5 other test classes which create a build directory directly in the module. 4 of them are looking for a system property test.cache.data, but default to build/test/cache.

src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
src/test/java/org/apache/hadoop/hdfs/util/TestCombinedHostsFileReader.java
src/test/java/org/apache/hadoop/hdfs/TestFileAppendRestart.java
src/test/java/org/apache/hadoop/hdfs/TestDFSShellGenericOptions.java
src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
src/test/java/org/apache/hadoop/hdfs/TestPersistBlocks.java

[hadoop-yetus]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 44s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 1s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	test4tests	0m 0s		The patch appears to include 13 new or modified test files.
			_ trunk Compile Tests _
+1 :green_heart:	mvninstall	49m 44s		trunk passed
+1 :green_heart:	compile	1m 29s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 :green_heart:	compile	1m 21s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 :green_heart:	checkstyle	1m 10s		trunk passed
+1 :green_heart:	mvnsite	1m 31s		trunk passed
+1 :green_heart:	javadoc	1m 9s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 :green_heart:	javadoc	1m 33s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 :green_heart:	spotbugs	3m 30s		trunk passed
+1 :green_heart:	shadedclient	27m 0s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	1m 34s		the patch passed
+1 :green_heart:	compile	1m 25s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 :green_heart:	javac	1m 25s		the patch passed
+1 :green_heart:	compile	1m 14s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 :green_heart:	javac	1m 14s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 54s		hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 257 unchanged - 1 fixed = 257 total (was 258)
+1 :green_heart:	mvnsite	1m 26s		the patch passed
+1 :green_heart:	javadoc	0m 51s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 :green_heart:	javadoc	1m 27s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 :green_heart:	spotbugs	3m 34s		the patch passed
+1 :green_heart:	shadedclient	27m 9s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 :x:	unit	206m 18s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 :green_heart:	asflicense	0m 41s		The patch does not generate ASF License warnings.
		333m 12s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.blockmanagement.TestBlockTokenWithShortCircuitRead

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/6/artifact/out/Dockerfile
GITHUB PR	https://github.com/apache/hadoop/pull/4835
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 7d7fc2526c95 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 495b92d1cb8127aeda442f5261516b3083f0816f
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/6/testReport/
Max. process+thread count	3969 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 28s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 1s		codespell was not available.
+0 :ok:	detsecrets	0m 1s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	test4tests	0m 0s		The patch appears to include 13 new or modified test files.
			_ trunk Compile Tests _
+1 :green_heart:	mvninstall	40m 23s		trunk passed
+1 :green_heart:	compile	1m 6s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 :green_heart:	compile	0m 56s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 :green_heart:	checkstyle	0m 51s		trunk passed
+1 :green_heart:	mvnsite	1m 4s		trunk passed
+1 :green_heart:	javadoc	0m 53s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 :green_heart:	javadoc	1m 19s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 :green_heart:	spotbugs	2m 38s		trunk passed
+1 :green_heart:	shadedclient	30m 2s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	0m 53s		the patch passed
+1 :green_heart:	compile	0m 55s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 :green_heart:	javac	0m 55s		the patch passed
+1 :green_heart:	compile	0m 49s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 :green_heart:	javac	0m 49s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
-0 :warning:	checkstyle	0m 45s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 257 unchanged - 1 fixed = 258 total (was 258)
+1 :green_heart:	mvnsite	0m 55s		the patch passed
+1 :green_heart:	javadoc	0m 39s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 :green_heart:	javadoc	1m 20s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 :green_heart:	spotbugs	2m 22s		the patch passed
+1 :green_heart:	shadedclient	28m 15s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 :x:	unit	210m 37s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 :green_heart:	asflicense	0m 35s		The patch does not generate ASF License warnings.
		329m 21s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.TestErasureCodingPoliciesWithRandomECPolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/7/artifact/out/Dockerfile
GITHUB PR	https://github.com/apache/hadoop/pull/4835
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux b16bcd1b7bd5 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / e9103cf75824aab1a95e8fa9062baf660cbb5fdc
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/7/testReport/
Max. process+thread count	3550 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/7/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 41s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 1s		codespell was not available.
+0 :ok:	detsecrets	0m 1s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	test4tests	0m 0s		The patch appears to include 13 new or modified test files.
			_ trunk Compile Tests _
+1 :green_heart:	mvninstall	47m 30s		trunk passed
+1 :green_heart:	compile	1m 39s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 :green_heart:	compile	1m 24s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 :green_heart:	checkstyle	1m 10s		trunk passed
+1 :green_heart:	mvnsite	1m 39s		trunk passed
+1 :green_heart:	javadoc	1m 19s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 :green_heart:	javadoc	1m 50s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 :green_heart:	spotbugs	3m 20s		trunk passed
+1 :green_heart:	shadedclient	40m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	1m 18s		the patch passed
+1 :green_heart:	compile	1m 21s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 :green_heart:	javac	1m 21s		the patch passed
+1 :green_heart:	compile	1m 15s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 :green_heart:	javac	1m 15s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	1m 5s		hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 257 unchanged - 1 fixed = 257 total (was 258)
+1 :green_heart:	mvnsite	1m 23s		the patch passed
+1 :green_heart:	javadoc	0m 56s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 :green_heart:	javadoc	1m 40s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 :green_heart:	spotbugs	3m 47s		the patch passed
+1 :green_heart:	shadedclient	41m 29s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 :green_heart:	unit	217m 58s		hadoop-hdfs in the patch passed.
+1 :green_heart:	asflicense	0m 52s		The patch does not generate ASF License warnings.
		374m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/8/artifact/out/Dockerfile
GITHUB PR	https://github.com/apache/hadoop/pull/4835
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux ea159205f3e0 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 12bcff4c81386889bd2291cbb9eecfa33012770e
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/8/testReport/
Max. process+thread count	3234 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4835/8/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[xinglin]

@steveloughran Is there anything you are waining for or are we ready to merge this PR? I want it to be merged soon! Thanks,

[xinglin]

@steveloughran/ @goiri could we merge this PR? Thanks,

[goiri]

I tested this a bit and it seems safe. Let's merge.

[xinglin]

Thanks @goiri for moving this PR forward!

[snmvaughan]

Thanks!