logging-flume icon indicating copy to clipboard operation
logging-flume copied to clipboard

FLUME-3354. Fix compilation error against Hive 2.3.5 and above.

Open iwasakims opened this issue 6 years ago • 2 comments

iwasakims avatar Feb 14 '20 04:02 iwasakims

Do we need to do any testing against new Hive features?

I would like to focus on whether existing use case works or not here. All unit tests of flume-hive-sink passed on my local. I ran the tests on flume-1.9 branch since trunk does not compile on flume-thrift-source (which is not related to this fix).

iwasakims avatar Feb 16 '20 13:02 iwasakims

Did you attempt to compile this against Hive3?

No. I'm working on this for BIGTOP-3123 which assumes Hadoop 2.10.0. Hive 3.x does not support Hadoop 2.x.

We are planning to upgrade Hadoop to 3.x on the release after the next. We need to upgrade Hive to 3.x for that. When I tried to compile Flume with -Dhadoop.version=3.2.1 -Dhive.version=3.1.2 , there was compilation error on flume-hdfs-sink. I will file a JIRA later if necessary.

iwasakims avatar Feb 16 '20 13:02 iwasakims