云瞻
云瞻
https://github.com/azkaban/azkaban-plugins/issues/267#issuecomment-374497146 This has a problem similar to mine, but I have also configured this parameter, but it has no effect
My Job conf.spark.yarn.archive=hdfs:///hdp/apps/3.1.0.0-78/spark2/spark2-hdp-hive-archive.tar.gz dependencies= driver-library-path=/usr/local/lib num-executors=20 conf.spark.history.fs.logDirectory=hdfs:///spark2-history/ conf.spark.driver.memory=4g execution-jar=hdfs:///spark-jar/test/layer_spark_20201019105827.jar conf.spark.executor.cores=4 conf.spark.yarn.jars=/usr/hdp/current/spark2-client/jars/*.jar conf.spark.eventLog.enabled=false type=spark conf.spark.files=file:///usr/hdp/current/spark2-client/conf/hive-site.xml conf.spark.dynamicAllocation.executorIdleTimeout=60 conf.spark.eventLog.dir=hdfs:///logs/spark-event-logs class=com.app.datacenter.main.MainSparkTest executor-memory=2G conf.spark.executor.memory=12G conf.spark.yarn.am.extraJavaOptions=-Dhdp.version=3.1.0.0-78 conf.spark.driver.extraJavaOptions=-Dhdp.version=3.1.0.0-78 params=$date() master=yarn verbose=false deploy-mode=cluster retries=0 conf.spark.dynamicAllocation.sustainedSchedulerBacklogTimeout=5 name=yarn_test conf.spark.dynamicAllocation.schedulerBacklogTimeout=1 queue=product
直接下载的代码是用mongo老版本方式mongodb-cr认证的,而新一点的mongo3以后默认都是scram-sha-1校验,所以过不去。可以自己编译一下最近代码。
> Please provide the Click House environment? clickhouse ver: 21.8.4.51 I can execute it through the earlier version of waterdrop-1.5.0,However, the above error reports were encountered in Seatunnel-2.3.0.