qingkuo han

spark rdma IBV_WC_WR_FLUSH_ERR

4

When I run Spark on yarn Spark2.1.0 Hadoop2.7.3 My spark task is correct But when my data is big,spark RdmaShuffleManager got error ,please check attch! Spark rdma conf: spark.driver.extraClassPath /home/bigdata/local/spark-rdma-3.1-for-spark-2.1.0-jar-with-dependencies.jar...

tez get job error

1

10-30-2018 12:28:33 ERROR [dr-el-executor-thread-1] com.linkedin.drelephant.ElephantRunner : Index: 0, Size: 0 10-30-2018 12:28:33 ERROR [dr-el-executor-thread-1] com.linkedin.drelephant.ElephantRunner : java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.rangeCheck(ArrayList.java:653) at java.util.ArrayList.get(ArrayList.java:429) at com.linkedin.drelephant.tez.fetchers.TezFetcher.fetchData(TezFetcher.java:125) at com.linkedin.drelephant.tez.fetchers.TezFetcher.fetchData(TezFetcher.java:40) at...

10-11-2018 18:47:53 INFO [Auto Tuner Thread] com.linkedin.drelephant.tuning.BaselineComputeUtil : Fetching jobs for which baseline metrics need to be computed 10-11-2018 18:47:53 INFO [Auto Tuner Thread] com.linkedin.drelephant.tuning.BaselineComputeUtil : Baseline computation complete 10-11-2018...