FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

[Gs928]

Apache Iceberg version

0.12.1

Query engine

Hive

Please describe the bug 🐞

hive> insert into test_iceberg_tbl1 values (1,"ggs",18,"20211212");

WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. Query ID = root_20220807223635_9a12c716-f280-4a76-a76c-1f5440ccc2e7 Total jobs = 1 Launching Job 1 out of 1 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_1659792265699_0058, Tracking URL = http://master:8088/proxy/application_1659792265699_0058/ Kill Command = /root/Hadooptools/hadoop-2.7.7/bin/hadoop job -kill job_1659792265699_0058 Hadoop job information for Stage-3: number of mappers: 1; number of reducers: 0 2022-08-07 22:36:53,130 Stage-3 map = 0%, reduce = 0% 2022-08-07 22:37:05,762 Stage-3 map = 100%, reduce = 0%, Cumulative CPU 4.6 sec MapReduce Total cumulative CPU time: 4 seconds 600 msec Ended Job = job_1659792265699_0058 with errors Error during job, obtaining debugging information... Examining task ID: task_1659792265699_0058_m_000000 (and more) from job job_1659792265699_0058 FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce Jobs Launched: Stage-Stage-3: Map: 1 Cumulative CPU: 4.6 sec HDFS Read: 115812 HDFS Write: 3854 FAIL Total MapReduce CPU Time Spent: 4 seconds 600 msec

An error was reported inserting data. However, it can be successfully executed when “set hive.exec.mode.local.auto=true;”. How can I fix it

[kbendick]

The Number of reduce tasks is set to 0 since there's no reduce operator is something you can ignore. That's just saying that this is a map only job.

What about this here?

An error was reported inserting data. However, it can be successfully executed when “set hive.exec.mode.local.auto=true;”.

Did you try set hive.exec.mode.local.auto=true; -- as mentioned in the error -- and did that work?

Also, can you provide the Iceberg related configuration for the table and for the job / session? Did you run the add jar command and the other steps from https://iceberg.apache.org/docs/latest/hive/?

[kbendick]

To be more precise, the 0.12.1 Iceberg docs on Hive are here: https://iceberg.apache.org/docs/0.12.1/hive/

If there's anymore error stack trace, that would be helpful. As well as the configuration and set up.

You also have to set engine.hive.enabled on the table that's being written to. See the docs linked above for more info on all of the steps that need to be run such as session properties to set and jars to be added and configuring the catalog properly.

[pvary]

As @kbendick mentioned, there should be an exception in the logs for the application. It would be great to check the application log for job_1659792265699_0058.

Also it would be great to have the exact Hive version as well.

Thanks, Peter

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions[bot]]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'