HBASE-29386: SnapshotProcedure and EnableTableProcedure can cause a deadlock

[hgromer]

[hgromer]

@rmdmattingly @ndimiduk @charlesconnell @sidkhillon @krconv

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 28s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+0 :ok:	mvndep	0m 15s		Maven dependency ordering for branch
+1 :green_heart:	mvninstall	3m 19s		master passed
+1 :green_heart:	compile	3m 36s		master passed
+1 :green_heart:	checkstyle	0m 46s		master passed
+1 :green_heart:	spotbugs	2m 0s		master passed
+1 :green_heart:	spotless	0m 50s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+0 :ok:	mvndep	0m 12s		Maven dependency ordering for patch
+1 :green_heart:	mvninstall	3m 5s		the patch passed
+1 :green_heart:	compile	3m 41s		the patch passed
+1 :green_heart:	javac	3m 41s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 46s		the patch passed
+1 :green_heart:	spotbugs	2m 11s		the patch passed
+1 :green_heart:	hadoopcheck	12m 24s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 :green_heart:	spotless	0m 46s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 :green_heart:	asflicense	0m 18s		The patch does not generate ASF License warnings.
		42m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux a5f316c1394b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 18bd20f9d4d9d9f0b83017a1adbf05066f65ae68
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-procedure hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/1/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 27s		Docker mode activated.
-0 :warning:	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+0 :ok:	mvndep	0m 16s		Maven dependency ordering for branch
+1 :green_heart:	mvninstall	3m 18s		master passed
+1 :green_heart:	compile	1m 13s		master passed
+1 :green_heart:	javadoc	0m 40s		master passed
+1 :green_heart:	shadedjars	6m 6s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 :ok:	mvndep	0m 13s		Maven dependency ordering for patch
+1 :green_heart:	mvninstall	3m 9s		the patch passed
+1 :green_heart:	compile	1m 12s		the patch passed
+1 :green_heart:	javac	1m 12s		the patch passed
+1 :green_heart:	javadoc	0m 38s		the patch passed
+1 :green_heart:	shadedjars	6m 2s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 :green_heart:	unit	1m 32s		hbase-procedure in the patch passed.
+1 :green_heart:	unit	211m 22s		hbase-server in the patch passed.
		240m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 11495b844953 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 18bd20f9d4d9d9f0b83017a1adbf05066f65ae68
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/1/testReport/
Max. process+thread count	5289 (vs. ulimit of 30000)
modules	C: hbase-procedure hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[charlesconnell]

Another option, which is simpler to reason about, is to simply fail the SnapshotProcedure if the table isn't in a state it can handle. This was my approach in HBASE-29315, for the exact same reasons you're encountering here. I couldn't usefully "sleep" a SplitTableRegionProcedure because it used holdLock=true. I think it would be good to agree on a standard of what to do in these situations.

Failing a procedure is simpler and doesn't introduce more edge cases in the procedure executor state machine. However, obviously, it's a better user experience if your procedures get executed eventually.

[hgromer]

Another option, which is simpler to reason about, is to simply fail the SnapshotProcedure if the table isn't in a state it can handle. This was my approach in HBASE-29315, for the exact same reasons you're encountering here. I couldn't usefully "sleep" a SplitTableRegionProcedure because it used holdLock=true. I think it would be good to agree on a standard of what to do in these situations.

Failing a procedure is simpler and doesn't introduce more edge cases in the procedure executor state machine. However, obviously, it's a better user experience if your procedures get executed eventually.

Agreed. The reason I opted for supsending the procedure is because this is already something we do if we are splitting or merging regions. So this procedure will already suspend and resume in the case the table state isn't optimal.

I don't think I have a very strong opinion one way or another, though. Enabling/disabling tables happens often enough that it might be fine to simply fail and force the user to manually handle the failure.

cc @Apache9 Don't know if you have any strong opinions here either way.

[rmdmattingly]

Enabling/disabling tables happens often enough that it might be fine to simply fail and force the user to manually handle the failure.

Do you mean "happens ~often~ infrequently enough"? If so, I agree that snapshotting a disabled table should be a pretty exceptional request, and so it is fine to err on the side of simplicity and just fail the procedure

[hgromer]

Enabling/disabling tables happens often enough that it might be fine to simply fail and force the user to manually handle the failure.

Do you mean "happens ~often~ infrequently enough"? If so, I agree that snapshotting a disabled table should be a pretty exceptional request, and so it is fine to err on the side of simplicity and just fail the procedure

Yes I did; I am happy to fail the procedure if that's the general consensus. That being said, I do think modern backups start to stress this system out. Any call to the BackupAdmin triggers an enable table procedure, so we're more likely to get into this state.

If we are okay with failing, then the user will have to manually kick off another snapshot on failure.

Another possible solution would be to reset the state of the snapshot procedure, so that it needs to run from the beginning

if (tableState.isEnabled()) {
  // action 
} else if (tableState.isDisabled) {
  // action
} else {
  // set up suspension timeout/persistence
  setNextState(SnapshotState.SNAPSHOT_PREPARE);
}

[hgromer]

going to proceed with the strategy of simply failing

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 28s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 23s		master passed
+1 :green_heart:	compile	3m 20s		master passed
+1 :green_heart:	checkstyle	0m 38s		master passed
+1 :green_heart:	spotbugs	1m 36s		master passed
+1 :green_heart:	spotless	0m 49s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 7s		the patch passed
+1 :green_heart:	compile	3m 15s		the patch passed
+1 :green_heart:	javac	3m 15s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 37s		the patch passed
+1 :green_heart:	spotbugs	1m 43s		the patch passed
+1 :green_heart:	hadoopcheck	12m 14s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 :green_heart:	spotless	0m 46s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 :green_heart:	asflicense	0m 10s		The patch does not generate ASF License warnings.
		40m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 5a56b88b319c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 31d51955b8ec2bef0cda620c842704e48ca06137
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/2/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 28s		Docker mode activated.
-0 :warning:	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 22s		master passed
+1 :green_heart:	compile	1m 5s		master passed
+1 :green_heart:	javadoc	0m 29s		master passed
+1 :green_heart:	shadedjars	6m 14s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 16s		the patch passed
+1 :green_heart:	compile	0m 59s		the patch passed
+1 :green_heart:	javac	0m 59s		the patch passed
+1 :green_heart:	javadoc	0m 28s		the patch passed
+1 :green_heart:	shadedjars	6m 6s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 :x:	unit	211m 40s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		238m 48s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux eccc001e1c95 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 31d51955b8ec2bef0cda620c842704e48ca06137
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/2/testReport/
Max. process+thread count	5273 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/2/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 38s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 :green_heart:	mvninstall	4m 37s		master passed
+1 :green_heart:	compile	4m 49s		master passed
+1 :green_heart:	checkstyle	0m 50s		master passed
+1 :green_heart:	spotbugs	2m 6s		master passed
+1 :green_heart:	spotless	1m 11s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	4m 9s		the patch passed
+1 :green_heart:	compile	4m 18s		the patch passed
+1 :green_heart:	javac	4m 18s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 42s		the patch passed
+1 :green_heart:	spotbugs	2m 25s		the patch passed
+1 :green_heart:	hadoopcheck	15m 5s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 :x:	spotless	0m 58s		patch has 29 errors when running spotless:check, run spotless:apply to fix.
			_ Other Tests _
+1 :green_heart:	asflicense	0m 14s		The patch does not generate ASF License warnings.
		51m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux f4853626e3e9 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 5b6ad478212b6c97434828be09ba126041b87401
Default Java	Eclipse Adoptium-17.0.11+9
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/3/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count	89 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/3/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 29s		Docker mode activated.
-0 :warning:	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 13s		master passed
+1 :green_heart:	compile	0m 58s		master passed
+1 :green_heart:	javadoc	0m 28s		master passed
+1 :green_heart:	shadedjars	6m 5s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 9s		the patch passed
+1 :green_heart:	compile	0m 57s		the patch passed
+1 :green_heart:	javac	0m 57s		the patch passed
+1 :green_heart:	javadoc	0m 26s		the patch passed
+1 :green_heart:	shadedjars	6m 2s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 :x:	unit	211m 38s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		237m 37s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux ed03acff7815 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 5b6ad478212b6c97434828be09ba126041b87401
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/3/testReport/
Max. process+thread count	5234 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/3/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 26s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 7s		master passed
+1 :green_heart:	compile	3m 19s		master passed
+1 :green_heart:	checkstyle	0m 36s		master passed
+1 :green_heart:	spotbugs	1m 35s		master passed
+1 :green_heart:	spotless	0m 45s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 2s		the patch passed
+1 :green_heart:	compile	3m 21s		the patch passed
+1 :green_heart:	javac	3m 21s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 37s		the patch passed
+1 :green_heart:	spotbugs	1m 41s		the patch passed
+1 :green_heart:	hadoopcheck	12m 13s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 :green_heart:	spotless	0m 45s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 :green_heart:	asflicense	0m 9s		The patch does not generate ASF License warnings.
		39m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux eaf376fc4bc8 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 937204375e22dffb465dab72bf5dc2d9daf9e313
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	84 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/4/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 28s		Docker mode activated.
-0 :warning:	yetus	0m 2s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 24s		master passed
+1 :green_heart:	compile	0m 59s		master passed
+1 :green_heart:	javadoc	0m 26s		master passed
+1 :green_heart:	shadedjars	6m 2s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 7s		the patch passed
+1 :green_heart:	compile	0m 57s		the patch passed
+1 :green_heart:	javac	0m 57s		the patch passed
+1 :green_heart:	javadoc	0m 28s		the patch passed
+1 :green_heart:	shadedjars	6m 3s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 :x:	unit	210m 32s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		237m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux ddbd13f3d69c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 937204375e22dffb465dab72bf5dc2d9daf9e313
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/4/testReport/
Max. process+thread count	5970 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/4/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 26s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 6s		master passed
+1 :green_heart:	compile	3m 22s		master passed
+1 :green_heart:	checkstyle	0m 37s		master passed
+1 :green_heart:	spotbugs	1m 35s		master passed
+1 :green_heart:	spotless	0m 46s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 9s		the patch passed
+1 :green_heart:	compile	3m 24s		the patch passed
+1 :green_heart:	javac	3m 24s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 37s		the patch passed
+1 :green_heart:	spotbugs	1m 43s		the patch passed
+1 :green_heart:	hadoopcheck	12m 8s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 :x:	spotless	0m 38s		patch has 29 errors when running spotless:check, run spotless:apply to fix.
			_ Other Tests _
+1 :green_heart:	asflicense	0m 9s		The patch does not generate ASF License warnings.
		39m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 00d7ba8b3f5f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 4143909ff899b9aa3c36092a3b655312e5ac9df6
Default Java	Eclipse Adoptium-17.0.11+9
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/5/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count	84 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/5/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[hgromer]

When implementing this SnapshotProcedure, we decided to use shared lock and holdLock = true to prevent other table procedure jumps in in the middle of our execution while not hurting the availability because exclusive lock will also prevent region assigning.

In HBASE-28683, we introduced a new way to only allow one procedure to run at the same time for the same table, so maybe a possible way is to make SnapshotProcedure also acquire exclusive lock and set holdLock = false, so it will not be executed at the same time with Enable and Disable procedure.

Thanks.

That's good to know, thank you for the context. The issue here is that the EnableTableProcedure will release the lock after it's subprocedures finish, which allows the SnapshotProcedure to execute on a table that is enabling. The SnapshotProcedure then gets stuck continuously re-running the same state over again, and will refuse to release the lock, creating a deadlock

With all this said, I think it makes the most sense, for simplicity's sake, to fail the procedure if the table is in an invalid state.

[Apache-HBase]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 29s		Docker mode activated.
-0 :warning:	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 11s		master passed
+1 :green_heart:	compile	0m 59s		master passed
+1 :green_heart:	javadoc	0m 27s		master passed
+1 :green_heart:	shadedjars	6m 5s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 3s		the patch passed
+1 :green_heart:	compile	0m 57s		the patch passed
+1 :green_heart:	javac	0m 57s		the patch passed
+1 :green_heart:	javadoc	0m 27s		the patch passed
+1 :green_heart:	shadedjars	5m 57s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 :x:	unit	211m 59s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		238m 0s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/5/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux a23e1a60072c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 4143909ff899b9aa3c36092a3b655312e5ac9df6
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/5/testReport/
Max. process+thread count	5332 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/5/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 27s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 9s		master passed
+1 :green_heart:	compile	3m 27s		master passed
+1 :green_heart:	checkstyle	0m 38s		master passed
+1 :green_heart:	spotbugs	1m 36s		master passed
+1 :green_heart:	spotless	0m 48s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 9s		the patch passed
+1 :green_heart:	compile	3m 13s		the patch passed
+1 :green_heart:	javac	3m 13s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 37s		the patch passed
+1 :green_heart:	spotbugs	1m 42s		the patch passed
+1 :green_heart:	hadoopcheck	12m 9s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 :green_heart:	spotless	0m 45s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 :green_heart:	asflicense	0m 9s		The patch does not generate ASF License warnings.
		39m 27s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 2046f527496b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / a33389edf0504be46b9487ef5943a409c652506e
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	83 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/6/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 29s		Docker mode activated.
-0 :warning:	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 22s		master passed
+1 :green_heart:	compile	0m 58s		master passed
+1 :green_heart:	javadoc	0m 28s		master passed
+1 :green_heart:	shadedjars	6m 1s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 9s		the patch passed
+1 :green_heart:	compile	0m 58s		the patch passed
+1 :green_heart:	javac	0m 58s		the patch passed
+1 :green_heart:	javadoc	0m 27s		the patch passed
+1 :green_heart:	shadedjars	6m 3s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 :green_heart:	unit	234m 52s		hbase-server in the patch passed.
		262m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/6/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 1a1979d71c37 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / a33389edf0504be46b9487ef5943a409c652506e
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/6/testReport/
Max. process+thread count	5249 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/6/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache9]

When implementing this SnapshotProcedure, we decided to use shared lock and holdLock = true to prevent other table procedure jumps in in the middle of our execution while not hurting the availability because exclusive lock will also prevent region assigning. In HBASE-28683, we introduced a new way to only allow one procedure to run at the same time for the same table, so maybe a possible way is to make SnapshotProcedure also acquire exclusive lock and set holdLock = false, so it will not be executed at the same time with Enable and Disable procedure. Thanks.

That's good to know, thank you for the context. The issue here is that the EnableTableProcedure will release the lock after it's subprocedures finish, which allows the SnapshotProcedure to execute on a table that is enabling. The SnapshotProcedure then gets stuck continuously re-running the same state over again, and will refuse to release the lock, creating a deadlock

With all this said, I think it makes the most sense, for simplicity's sake, to fail the procedure if the table is in an invalid state.

Using the mechanism in HBASE-28683, SnapshotProcedure can not be executed together with EnableTableProcedure/DisabledTableProcedure together, which could also solve the problem, and I think it is more stable. I'm not sure whether ModifyTableProcedure could also affect SnapshotProcedure if it jumps in just in the middle of the execution...

[hgromer]

When implementing this SnapshotProcedure, we decided to use shared lock and holdLock = true to prevent other table procedure jumps in in the middle of our execution while not hurting the availability because exclusive lock will also prevent region assigning. In HBASE-28683, we introduced a new way to only allow one procedure to run at the same time for the same table, so maybe a possible way is to make SnapshotProcedure also acquire exclusive lock and set holdLock = false, so it will not be executed at the same time with Enable and Disable procedure. Thanks.

That's good to know, thank you for the context. The issue here is that the EnableTableProcedure will release the lock after it's subprocedures finish, which allows the SnapshotProcedure to execute on a table that is enabling. The SnapshotProcedure then gets stuck continuously re-running the same state over again, and will refuse to release the lock, creating a deadlock With all this said, I think it makes the most sense, for simplicity's sake, to fail the procedure if the table is in an invalid state.

Using the mechanism in HBASE-28683, SnapshotProcedure can not be executed together with EnableTableProcedure/DisabledTableProcedure together, which could also solve the problem, and I think it is more stable. I'm not sure whether ModifyTableProcedure could also affect SnapshotProcedure if it jumps in just in the middle of the execution...

Yes, I'm able to avoid the deadlock if the SnapshotProcedure both sets holdLock = true and also yields after procedure execution. Essentially allowing both SnapshotProcedure and EnableTableProcedure to execute without the need to throw any sort of yield / suspended exception (which was my initial implementation).

This is certainly a cleaner implementation, though it means we can interleave other table procedures after cycles of the SnapshotProcedure.

You hint at this here

I'm not sure whether ModifyTableProcedure could also affect SnapshotProcedure if it jumps in just in the middle of the execution...

and I believe that it would be problematic because you may snapshot regions multiple times, regions may move or go offline, so you may miss out on snapshotting regions.

My initial implementation prevented this by only allowing SNAPSHOT_WRITE_SNAPSHOT_INFO to release the lock (when it's still safe to do so). However, I don't think it's safe to yield the lock after any cycle.

After some thought, I think we still have two solutions here:

1. My initial implementation, which throws an exception that indicates we should release the lock from within SNAPSHOT_WRITE_SNAPSHOT_INFO
2. Fail the procedure

It seems the consensus is to avoid complicating the procedure and fail if we encounter the table in an invalid state, so I'm leaning towards 2 thought admittedly I'm a bit torn

[Apache9]

This is certainly a cleaner implementation, though it means we can interleave other table procedures after cycles of the SnapshotProcedure.

FWIW, if a SnapshotProcedure starts, no other table procedures which need a exclusive lock can be executed...

And what I mean is that, we can reuse the mechanism introduced in HBASE-28683 to simply fix the problem.

Just change the code here

https://github.com/apache/hbase/blob/64c582fe9a846b2f5e07b180cc6fae77431d726e/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TableQueue.java#L62

Make Snapshot also return true, and change SnapshotProcedure's acquireLock method to also require exclusive lock, change holdLock to return false, then we are safe.

Thanks.

[hgromer]

This is certainly a cleaner implementation, though it means we can interleave other table procedures after cycles of the SnapshotProcedure.

FWIW, if a SnapshotProcedure starts, no other table procedures which need a exclusive lock can be executed...

And what I mean is that, we can reuse the mechanism introduced in HBASE-28683 to simply fix the problem.

Just change the code here

https://github.com/apache/hbase/blob/64c582fe9a846b2f5e07b180cc6fae77431d726e/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TableQueue.java#L62

Make Snapshot also return true, and change SnapshotProcedure's acquireLock method to also require exclusive lock, change holdLock to return false, then we are safe.

Thanks.

Ah okay, I understand. This does prevent us from being able to take multiple snapshots of the table, as mentioned in this comment but that seems ok

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 29s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 59s		master passed
+1 :green_heart:	compile	4m 3s		master passed
+1 :green_heart:	checkstyle	0m 42s		master passed
+1 :green_heart:	spotbugs	1m 44s		master passed
+1 :green_heart:	spotless	0m 53s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 45s		the patch passed
+1 :green_heart:	compile	3m 30s		the patch passed
+1 :green_heart:	javac	3m 30s		the patch passed
+1 :green_heart:	blanks	0m 1s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 40s		the patch passed
+1 :green_heart:	spotbugs	2m 1s		the patch passed
+1 :green_heart:	hadoopcheck	12m 28s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 :green_heart:	spotless	0m 44s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 :green_heart:	asflicense	0m 10s		The patch does not generate ASF License warnings.
		42m 48s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/7/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 364abec9f6d4 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 8adc8bf77e3d8e98606d8a42c852841ff2038ee6
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	83 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/7/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 29s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 45s		master passed
+1 :green_heart:	compile	3m 27s		master passed
+1 :green_heart:	checkstyle	0m 38s		master passed
+1 :green_heart:	spotbugs	1m 38s		master passed
+1 :green_heart:	spotless	0m 49s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 7s		the patch passed
+1 :green_heart:	compile	3m 16s		the patch passed
+1 :green_heart:	javac	3m 16s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 36s		the patch passed
+1 :green_heart:	spotbugs	1m 42s		the patch passed
+1 :green_heart:	hadoopcheck	12m 18s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 :green_heart:	spotless	0m 46s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 :green_heart:	asflicense	0m 10s		The patch does not generate ASF License warnings.
		41m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/8/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 9dd13ef03bae 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 01997a45c9f2a618a363ba6fc6ac6c6a9354cd49
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/8/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 33s		Docker mode activated.
-0 :warning:	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 55s		master passed
+1 :green_heart:	compile	0m 57s		master passed
+1 :green_heart:	javadoc	0m 29s		master passed
+1 :green_heart:	shadedjars	6m 12s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 7s		the patch passed
+1 :green_heart:	compile	0m 59s		the patch passed
+1 :green_heart:	javac	0m 59s		the patch passed
+1 :green_heart:	javadoc	0m 28s		the patch passed
+1 :green_heart:	shadedjars	6m 4s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 :x:	unit	226m 6s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		253m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/8/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 2c2a9570b591 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 01997a45c9f2a618a363ba6fc6ac6c6a9354cd49
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/8/testReport/
Max. process+thread count	5505 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/8/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache9]

TestSnapshotProcedureRIT.testTableInMergeWhileTakingSnapshot failed, I guess this is related to our changes here?

[hgromer]

TestSnapshotProcedureRIT.testTableInMergeWhileTakingSnapshot failed, I guess this is related to our changes here?

Yes must be related I am taking a look

[Apache-HBase]

:confetti_ball: +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 :green_heart:	dupname	0m 0s		No case conflicting files found.
+0 :ok:	codespell	0m 0s		codespell was not available.
+0 :ok:	detsecrets	0m 0s		detect-secrets was not available.
+1 :green_heart:	@author	0m 0s		The patch does not contain any @author tags.
+1 :green_heart:	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 54s		master passed
+1 :green_heart:	compile	3m 24s		master passed
+1 :green_heart:	checkstyle	0m 38s		master passed
+1 :green_heart:	spotbugs	1m 40s		master passed
+1 :green_heart:	spotless	0m 49s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 7s		the patch passed
+1 :green_heart:	compile	3m 17s		the patch passed
+1 :green_heart:	javac	3m 17s		the patch passed
+1 :green_heart:	blanks	0m 0s		The patch has no blanks issues.
+1 :green_heart:	checkstyle	0m 37s		the patch passed
+1 :green_heart:	spotbugs	1m 42s		the patch passed
+1 :green_heart:	hadoopcheck	12m 16s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 :green_heart:	spotless	0m 45s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 :green_heart:	asflicense	0m 10s		The patch does not generate ASF License warnings.
		40m 38s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/9/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 4e14747c481e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / f14a7cff03a1046a4eba2597e529844fa646c193
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	84 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/9/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

:broken_heart: -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 :ok:	reexec	0m 31s		Docker mode activated.
-0 :warning:	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 :green_heart:	mvninstall	3m 53s		master passed
+1 :green_heart:	compile	0m 58s		master passed
+1 :green_heart:	javadoc	0m 28s		master passed
+1 :green_heart:	shadedjars	6m 5s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 :green_heart:	mvninstall	3m 11s		the patch passed
+1 :green_heart:	compile	0m 57s		the patch passed
+1 :green_heart:	javac	0m 57s		the patch passed
+1 :green_heart:	javadoc	0m 27s		the patch passed
+1 :green_heart:	shadedjars	6m 2s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 :x:	unit	214m 55s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		242m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/9/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	https://github.com/apache/hbase/pull/7084
JIRA Issue	HBASE-29386
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 9a40d0fa1370 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / f14a7cff03a1046a4eba2597e529844fa646c193
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/9/testReport/
Max. process+thread count	5312 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7084/9/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.