HIVE-26367: Upgrade to tez 0.10.2

[abstractdog]

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

[ayushtkn]

Tried with my Hadoop PR and 2 tests failed, 1 doesn't seems related but 1 does can you give a check: http://ci.hive.apache.org/job/hive-precommit/job/PR-3279/12/testReport/junit/org.apache.hadoop.hive.llap.tezplugins/TestLlapTaskCommunicator/Testing___split_10___PostProcess___testFinishableStateUpdateFailure/

[abstractdog]

thanks @ayushtkn , I fixed TestLlapTaskCommunicator locally with a simple test change in the precommit I can see mass failures, e.g. in TestMmCompactorOnMr, where hive.log contains:

Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.fs.Path.compareTo(Lorg/apache/hadoop/fs/Path;)I
        at org.apache.hadoop.io.FileChunk.compareTo(FileChunk.java:82) ~[tez-runtime-library-0.10.2.jar:?]
        at org.apache.hadoop.io.FileChunk.compareTo(FileChunk.java:27) ~[tez-runtime-library-0.10.2.jar:?]
        at java.util.TreeMap.compare(TreeMap.java:1294) ~[?:1.8.0_292]
        at java.util.TreeMap.put(TreeMap.java:538) ~[?:1.8.0_292]
        at java.util.TreeSet.add(TreeSet.java:255) ~[?:1.8.0_292]
        at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.closeOnDiskFile(MergeManager.java:598) ~[tez-runtime-library-0.10.2.jar:0.10.2]
        at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MapOutput$DiskDirectMapOutput.commit(MapOutput.java:188) ~[tez-runtime-library-0.10.2.jar:0.10.2]
        at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.ShuffleScheduler.copySucceeded(ShuffleScheduler.java:605) ~[tez-runtime-library-0.10.2.jar:0.10.2]
        at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.FetcherOrderedGrouped.setupLocalDiskFetch(FetcherOrderedGrouped.java:739) ~[tez-runtime-library-0.10.2.jar:0.10.2]
        at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.FetcherOrderedGrouped.fetchNext(FetcherOrderedGrouped.java:183) ~[tez-runtime-library-0.10.2.jar:0.10.2]
        at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.FetcherOrderedGrouped.callInternal(FetcherOrderedGrouped.java:198) ~[tez-runtime-library-0.10.2.jar:0.10.2]
        at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.FetcherOrderedGrouped.callInternal(FetcherOrderedGrouped.java:61) ~[tez-runtime-library-0.10.2.jar:0.10.2]
        ... 7 more

is it something related to the fact that tez 0.10.2 was compiled against the newer hadoop? if it's only the case, we'll simply merge tez and hadoop upgrade together

UPDATE: checked with hadoop 3.3.1, and for instance TestMmCompactorOnMr issues have been gone

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the [email protected] list if the patch is in need of reviews.