HBASE-28879 Bump hbase-thirdparty to 4.1.9 #6295

ndimiduk · 2024-09-25T08:52:47Z

ndimiduk · 2024-09-25T09:14:50Z

FYI @Apache9 @apurtell The backport to branch-2.5 does not apply cleanly. The version of hbase-thirdparty on branch-2.5 is several patch released behind.

I think it would be wise to increment the version there as well, for the purpose of CVE mitigation.

ndimiduk · 2024-09-25T09:23:26Z

I talked myself into it. Here's a backport for branch-2.5.

ndimiduk · 2024-09-25T09:35:03Z

Oops nope, that a major version bump of the protobuf jar.

Apache9 · 2024-09-25T09:40:28Z

Thanks @ndimiduk for taking care of this.

There is an issue for this HBASE-28879.

ndimiduk · 2024-09-25T09:42:16Z

Perfect. Thanks.

NihalJain · 2024-09-25T10:45:39Z

pom.xml

@@ -931,7 +931,7 @@
        databind] must be kept in sync with the version of jackson-jaxrs-json-provider shipped in
        hbase-thirdparty.
    -->
-    <hbase-thirdparty.version>4.1.8</hbase-thirdparty.version>
+    <hbase-thirdparty.version>4.1.9</hbase-thirdparty.version>


we may also want to sync versions of error prone and netty4 with hbase-thirdparty

This is getting out of hand. See how many fix-up commits I had to push to various projects? We should be managing all these version numbers with a BOM import.

This blog post has a nice write-up on the idea, https://www.garretwilson.com/blog/2023/06/14/improve-maven-bom-pattern

I think that hbase-thirdparty could publish a BOM pom file that can be imported into any of the downstream hbase projects that make use of that release of hbase-thirdparty. That will centralize management of these dependencies in the hbase-thirdparty repo, and we won't have to play whack-a-mole in the main project poms.

Right, I was thinking about the same. Because it does not make sense to waste time doing this and still miss out on one or the another. We already have something similar for org.mockito:mockito-bom:

hbase/pom.xml

Line 1626 in 449c446

<artifactId>mockito-bom</artifactId>

We should file a jira to handle this. At least will have a better way of doing thirdparty change with a one liner change with next thirdparty release, which is how it should have been ideally.

I can help with implementing this, please lmk if you do not plan to fix this yourself.

Filed https://issues.apache.org/jira/browse/HBASE-28883

Updating error prone is not trivial, sometimes it may cause some compile errors since they may introduced some new check rules, so usually I will open a new issue for updating error prone.

And on netty4 dependencies, hbase does not depend on it directly since we use the shaded one in hbase thirdparty, it is mainly introduced by hadoop and zookeeper. And in hadoop 3.4.0, IIRC hadoop also shaded and relocated netty in their thirdparty jar, so maybe we need to discuss whether we will need to force netty dependencies in hbase, maybe just leave it as is.

And on netty4 dependencies, hbase does not depend on it directly since we use the shaded one in hbase thirdparty, it is mainly introduced by hadoop and zookeeper. And in hadoop 3.4.0, IIRC hadoop also shaded and relocated netty in their thirdparty jar, so maybe we need to discuss whether we will need to force netty dependencies in hbase, maybe just leave it as is.

So we need not necessarily align netty with thirdparty.
Same for error prone.

Filed https://issues.apache.org/jira/browse/HBASE-28883

Please see apache/hbase-thirdparty#124, NihalJain@0b51c71

Okay so we do not need to maintain alignment on netty or errorprone? Jackson and protobuf are the only two that require alignment?

Let me explain more clear.

For error prone, we'd better also bump the error prone to the same version with thirparty, but it is not trivial sometimes, and since hbase-thirdparty only depends on the annotation jar(even not shaded), so it is not likely to introduce any conflicts. So I think it is better to have a separated issue for bumping it, after upgrading the hbase-thirdparty.

For netty, since hbase does not depend on netty4 directly, we do not need to align the netty version with the one in hbase-thirdparty.
We maintain it in our pom is because the conflicts between zookeeper and hadoop. So if there are no CVEs for netty, we do not need to bump it in hbase. And after hadoop 3.4.0, since hadoop also shade netty(IIRC), maybe we even do not need to do this any more. If there are new CVEs for netty, maybe we just need to bump the zookeeper dependency?

Apache-HBase · 2024-09-27T14:01:27Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 39s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ master Compile Tests _
+0 🆗	mvndep	0m 16s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 9s		master passed
+1 💚	compile	8m 34s		master passed
+1 💚	spotless	0m 46s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 17s		Maven dependency ordering for patch
+1 💚	mvninstall	4m 9s		the patch passed
+1 💚	compile	9m 49s		the patch passed
-0 ⚠️	javac	9m 49s	/results-compile-javac-root.txt	root generated 10 new + 1205 unchanged - 7 fixed = 1215 total (was 1212)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	xmllint	0m 0s		No new issues.
+1 💚	hadoopcheck	12m 53s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚	spotless	1m 6s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 49s		The patch does not generate ASF License warnings.
		50m 57s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6295/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6295
Optional Tests	dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile
uname	Linux 714a99bf9835 5.4.0-192-generic #212-Ubuntu SMP Fri Jul 5 09:47:39 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `a37b900`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	192 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-examples . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6295/4/console
versions	git=2.34.1 maven=3.9.8 xmllint=20913
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-27T18:50:13Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 18s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+0 🆗	mvndep	0m 15s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 5s		master passed
+1 💚	compile	2m 44s		master passed
+1 💚	javadoc	2m 56s		master passed
+1 💚	shadedjars	6m 45s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 20s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 38s		the patch passed
+1 💚	compile	2m 43s		the patch passed
+1 💚	javac	2m 43s		the patch passed
+1 💚	javadoc	3m 34s		the patch passed
+1 💚	shadedjars	6m 38s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	300m 12s	/patch-unit-root.txt	root in the patch failed.
		339m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6295/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6295
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux be86163b9995 5.4.0-192-generic #212-Ubuntu SMP Fri Jul 5 09:47:39 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `a37b900`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6295/4/testReport/
Max. process+thread count	5610 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-examples . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6295/4/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

This was referenced Sep 25, 2024

Backport "HBASE-28879 Bump hbase-thirdparty to 4.1.9" to branch-3 #6296

Merged

Backport "HBASE-28879 Bump hbase-thirdparty to 4.1.9" to branch-2 #6297

Merged

Backport "HBASE-28879 Bump hbase-thirdparty to 4.1.9" to branch-2.6 #6298

Merged

ndimiduk mentioned this pull request Sep 25, 2024

Backport "Test with hbase-thirdparty 4.1.9RC0" to branch-2.5 #6299

Closed

ndimiduk changed the title ~~Test with hbase-thirdparty 4.1.9RC0~~ HBASE-28879 Bump hbase-thirdparty to 4.1.9 Sep 25, 2024

This comment has been minimized.

Sign in to view

NihalJain reviewed Sep 25, 2024

View reviewed changes

Apache9 approved these changes Sep 25, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

ndimiduk force-pushed the hbase-thirdparty-4.1.9rc0 branch from 9d9bb38 to c72cc80 Compare September 26, 2024 09:05

This comment has been minimized.

Sign in to view

ndimiduk added 5 commits September 27, 2024 14:36

Test with hbase-thirdparty 4.1.9RC0

a51c132

bump protobuf version in hbase-protocol-shaded and hbase-examples

7373efd

reformat protobuf comment

a8a25b6

spotless:apply

623cfb3

remove staging repository

a37b900

ndimiduk force-pushed the hbase-thirdparty-4.1.9rc0 branch from c72cc80 to a37b900 Compare September 27, 2024 12:37

ndimiduk marked this pull request as ready for review September 27, 2024 12:38

NihalJain approved these changes Sep 27, 2024

View reviewed changes

ndimiduk merged commit 788cf74 into apache:master Sep 30, 2024
1 check failed

ndimiduk deleted the hbase-thirdparty-4.1.9rc0 branch September 30, 2024 10:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-28879 Bump hbase-thirdparty to 4.1.9 #6295

HBASE-28879 Bump hbase-thirdparty to 4.1.9 #6295

ndimiduk commented Sep 25, 2024

ndimiduk commented Sep 25, 2024

ndimiduk commented Sep 25, 2024

ndimiduk commented Sep 25, 2024

Apache9 commented Sep 25, 2024

ndimiduk commented Sep 25, 2024

This comment has been minimized.

NihalJain Sep 25, 2024

ndimiduk Sep 25, 2024

NihalJain Sep 25, 2024

NihalJain Sep 25, 2024 •

edited

Loading

ndimiduk Sep 25, 2024

Apache9 Sep 25, 2024

Apache9 Sep 25, 2024

NihalJain Sep 25, 2024

ndimiduk Sep 26, 2024

Apache9 Sep 27, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Apache-HBase commented Sep 27, 2024

Apache-HBase commented Sep 27, 2024

HBASE-28879 Bump hbase-thirdparty to 4.1.9 #6295

HBASE-28879 Bump hbase-thirdparty to 4.1.9 #6295

Conversation

ndimiduk commented Sep 25, 2024

ndimiduk commented Sep 25, 2024

ndimiduk commented Sep 25, 2024

ndimiduk commented Sep 25, 2024

Apache9 commented Sep 25, 2024

ndimiduk commented Sep 25, 2024

This comment has been minimized.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NihalJain Sep 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Apache-HBase commented Sep 27, 2024

Apache-HBase commented Sep 27, 2024

NihalJain Sep 25, 2024 •

edited

Loading