bigtop icon indicating copy to clipboard operation
bigtop copied to clipboard

BIGTOP-3989: Bump HBase to 2.5.3

Open Akanksha-kedia opened this issue 2 years ago • 12 comments

BIGTOP-3989 Upgrade hbase version to 2.5.3

https://apache.osuosl.org/phoenix/ https://archive.apache.org/dist/phoenix/ https://apache.osuosl.org/hbase/ https://archive.apache.org/dist/hbase/

'hbase' { name = 'hbase' rpm_pkg_suffix = "" + bigtop.base_version.replace(".", "") relNotes = 'Apache HBase' version { base = '2.4.13'; pkg = base; release = 2 } tarball { destination = "${name}-${version.base}.tar.gz" source = "${name}-${version.base}-src.tar.gz" } url { download_path = "/$name/${version.base}/" site = "${apache.APACHE_MIRROR}/${download_path}" archive = "${apache.APACHE_ARCHIVE}/${download_path}" } }

Task :hbase-pkg Caching disabled for task ':hbase-pkg' because: Build cache is disabled Task ':hbase-pkg' is not up-to-date because: Task has not declared any outputs despite executing actions. :hbase-pkg (Thread[Execution worker for ':' Thread 2,5,main]) completed. Took 0.002 secs.

BUILD SUCCESSFUL in 26m 23s 6 actionable tasks: 6 executed

CentOS Linux(x86_64)

Akanksha-kedia avatar Sep 25 '23 15:09 Akanksha-kedia

@smallyao please review

Akanksha-kedia avatar Sep 25 '23 15:09 Akanksha-kedia

@smallyao @tsudukim please review

Akanksha-kedia avatar Oct 06 '23 06:10 Akanksha-kedia

@smallyao @tsudukim please review

Akanksha-kedia avatar Oct 06 '23 06:10 Akanksha-kedia

for 2.2.5: image

On removing the patch, able to successfully upgrade hbase version.

Task :hbase-pkg Caching disabled for task ':hbase-pkg' because: Build cache is disabled Task ':hbase-pkg' is not up-to-date because: Task has not declared any outputs despite executing actions. :hbase-pkg (Thread[Daemon worker Thread 3,5,main]) completed. Took 0.0 secs.

Akanksha-kedia avatar Oct 09 '23 08:10 Akanksha-kedia

hbase_3_3_0-2.5.5-6.el7.x86_64.rpm hbase_3_3_0-rest-2.5.5-6.el7.x86_64.rpm hbase_3_3_0-master-2.5.5-6.el7.x86_64.rpm hbase_3_3_0-thrift2-2.5.5-6.el7.x86_64.rpm hbase_3_3_0-regionserver-2.5.5-6.el7.x86_64.rpm hbase_3_3_0-thrift-2.5.5-6.el7.x86_64.rpm

Akanksha-kedia avatar Oct 09 '23 09:10 Akanksha-kedia

Hi @Akanksha-kedia As mentioned by @iwasakims in #1166 , You must build both RPM and DEB package and run smoke-tests of hbase at least

smallyao avatar Oct 13 '23 09:10 smallyao

Yes i have seen his msg.can you guide with the smoke test setup as hbase seems to be dependent on zookeeper.

On Fri, 13 Oct 2023 at 2:55 PM, yaolei @.***> wrote:

Hi @Akanksha-kedia https://github.com/Akanksha-kedia As mentioned by @iwasakims https://github.com/iwasakims in #1166 https://github.com/apache/bigtop/pull/1166 , You must build both RPM and DEB package and run smoke-tests of hbase at least

— Reply to this email directly, view it on GitHub https://github.com/apache/bigtop/pull/1182#issuecomment-1761203109, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVL2AZUS74J55HMYQ5PGGVLX7ECJVAVCNFSM6AAAAAA5GHSYJKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRRGIYDGMJQHE . You are receiving this because you were mentioned.Message ID: @.***>

Akanksha-kedia avatar Oct 13 '23 09:10 Akanksha-kedia

You can see #1174

smallyao avatar Oct 13 '23 11:10 smallyao

The current PR doesn't work, as follows:

$ curl -sL https://github.com/apache/bigtop/pull/1182.diff | git apply
$ git diff
diff --git a/bigtop.bom b/bigtop.bom
index 01fbdd74..bf51e049 100644
--- a/bigtop.bom
+++ b/bigtop.bom
@@ -162,7 +162,7 @@ bigtop {
       name    = 'hbase'
       rpm_pkg_suffix = "_" + bigtop.base_version.replace(".", "_")
       relNotes = 'Apache HBase'
-      version { base = '2.4.17'; pkg = base; release = 1 }
+      version { base = '2.5.3'; pkg = base; release = 1 }
       tarball { destination = "${name}-${version.base}.tar.gz"
                 source      = "${name}-${version.base}-src.tar.gz" }
       url     { download_path = "/$name/${version.base}/"
$ ./gradlew hbase-clean hbase-pkg

...

dpkg-source: error: LC_ALL=C patch -t -F 0 -N -p1 -u -V never -E -b -B .pc/patch0-HBASE-27065-2.4.13.diff/ --reject-file=- < hbase-2.5.3.orig._x04bj/debian/patches/patch0-HBASE-27065-2.4.13.diff subprocess returned exit status 1
patching file pom.xml
Reversed (or previously applied) patch detected!  Skipping patch.
13 out of 13 hunks ignored
dpkg-source: info: the patch has fuzz which is not allowed, or is malformed
dpkg-source: info: if patch 'patch0-HBASE-27065-2.4.13.diff' is correctly applied by quilt, use 'quilt refresh' to update it
dpkg-source: info: if the file is present in the unpacked source, make sure it is also present in the orig tarball
dpkg-buildpackage: error: dpkg-source -b . subprocess returned exit status 2

> Task :hbase-sdeb FAILED

FAILURE: Build failed with an exception.

On removing the patch, able to successfully upgrade hbase version.

If so, please include that deletion in your PR.

sekikn avatar Oct 15 '23 13:10 sekikn

The current PR doesn't work, as follows:

$ curl -sL https://github.com/apache/bigtop/pull/1182.diff | git apply
$ git diff
diff --git a/bigtop.bom b/bigtop.bom
index 01fbdd74..bf51e049 100644
--- a/bigtop.bom
+++ b/bigtop.bom
@@ -162,7 +162,7 @@ bigtop {
       name    = 'hbase'
       rpm_pkg_suffix = "_" + bigtop.base_version.replace(".", "_")
       relNotes = 'Apache HBase'
-      version { base = '2.4.17'; pkg = base; release = 1 }
+      version { base = '2.5.3'; pkg = base; release = 1 }
       tarball { destination = "${name}-${version.base}.tar.gz"
                 source      = "${name}-${version.base}-src.tar.gz" }
       url     { download_path = "/$name/${version.base}/"
$ ./gradlew hbase-clean hbase-pkg

...

dpkg-source: error: LC_ALL=C patch -t -F 0 -N -p1 -u -V never -E -b -B .pc/patch0-HBASE-27065-2.4.13.diff/ --reject-file=- < hbase-2.5.3.orig._x04bj/debian/patches/patch0-HBASE-27065-2.4.13.diff subprocess returned exit status 1
patching file pom.xml
Reversed (or previously applied) patch detected!  Skipping patch.
13 out of 13 hunks ignored
dpkg-source: info: the patch has fuzz which is not allowed, or is malformed
dpkg-source: info: if patch 'patch0-HBASE-27065-2.4.13.diff' is correctly applied by quilt, use 'quilt refresh' to update it
dpkg-source: info: if the file is present in the unpacked source, make sure it is also present in the orig tarball
dpkg-buildpackage: error: dpkg-source -b . subprocess returned exit status 2

> Task :hbase-sdeb FAILED

FAILURE: Build failed with an exception.

On removing the patch, able to successfully upgrade hbase version.

If so, please include that deletion in your PR.

please review

Akanksha-kedia avatar Oct 15 '23 15:10 Akanksha-kedia

Your second force-push rewrote HBase version to 2.5.5, so the latest PR doesn't work again.

$ curl -sL https://github.com/apache/bigtop/pull/1182.diff | git apply
$ git diff --stat
 bigtop-packages/src/common/hbase/patch0-HBASE-27065-2.4.13.diff | 251 ----------------------------------------------------
 bigtop.bom                                                      |   2 +-
 2 files changed, 1 insertion(+), 252 deletions(-)
$ git diff bigtop.bom
diff --git a/bigtop.bom b/bigtop.bom
index 01fbdd7..16141be 100644
--- a/bigtop.bom
+++ b/bigtop.bom
@@ -162,7 +162,7 @@ bigtop {
       name    = 'hbase'
       rpm_pkg_suffix = "_" + bigtop.base_version.replace(".", "_")
       relNotes = 'Apache HBase'
-      version { base = '2.4.17'; pkg = base; release = 1 }
+      version { base = '2.5.5'; pkg = base; release = 1 }
       tarball { destination = "${name}-${version.base}.tar.gz"
                 source      = "${name}-${version.base}-src.tar.gz" }
       url     { download_path = "/$name/${version.base}/"
$ ./gradlew hbase-clean hbase-pkg

...

+ cd hbase-2.5.5
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ echo 'Patch #0 (patch2-HBASE-27860-2.4.13.diff):'
Patch #0 (patch2-HBASE-27860-2.4.13.diff):
+ /usr/bin/patch -p1 --fuzz=0
+ /usr/bin/cat /home/vagrant/bigtop/build/hbase/rpm/SOURCES/patch2-HBASE-27860-2.4.13.diff
patching file hbase-shaded/pom.xml
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file hbase-shaded/pom.xml.rej


RPM build errors:
error: Bad exit status from /var/tmp/rpm-tmp.SoZ4qd (%prep)
    Bad exit status from /var/tmp/rpm-tmp.SoZ4qd (%prep)

FAILURE: Build failed with an exception.

I'd like you to test your PR locally before asking review, as @iwasakims and @smallyao already pointed out.

sekikn avatar Oct 15 '23 22:10 sekikn

In addition, I've just noticed that RANGER-4029 is included in the release notes of Ranger 2.4.0 but it's not merged into Ranger 2.4.0 actually. Is it possible to address this issue on the Ranger side? It it's difficult, I suggest we should stay HBase 2.4.17 in the 3.3.0 release.

sekikn avatar Oct 15 '23 23:10 sekikn