[Q&A]Ask any questions about using or deploying Apache DolphinScheduler

[debrachena]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Description

If you encounter any issues while using or deploying Apache DolphinScheduler, feel free to ask here. Please describe the problem clearly and logically, and if necessary, you can include images to illustrate your question. All community members are welcome to help with answering questions.

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[shuaihehe]

请问，3.1.0版本中，dolphinscheduler本身只有在master模块里有负载均衡算法，根据算法选中其中一台worker节点进行任务执行。想要去掉worker模块，使用k8s或者其他第三方进行任务调度执行，可以实现吗？我发现worker与master、zookeeper之间有好多通信，去掉worker模块以后这些通信怎么解决呢？可以给我指导指导吗？万分感谢！

[xmshdonghua]

3.2.1版本中，运行java的任务，选择主程序包和资源，运行时会报下面的错误，这个问题要如何解决？谢谢！

---

[INFO] 2024-03-30 12:31:32.735 +0800 - #!/bin/bash BASEDIR=$(cd dirname $0; pwd) cd $BASEDIR ${JAVA_HOME}/bin/java -classpath .:/data/bigdata/dolphinscheduler/exec/process/root/13044756119168/13099017794240_1/34/33:/data/bigdata/dolphinscheduler/exec/process/root/13044756119168/13099017794240_1/34/33//data/bigdata/dolphinscheduler/exec/process/root/13044756119168/13099017794240_1/34/33/task-scheduler-demo-0.0.1-SNAPSHOT.jar -jar /data/bigdata/dolphinscheduler/exec/process/root/13044756119168/13099017794240_1/34/33//data/bigdata/dolphinscheduler/exec/process/root/13044756119168/13099017794240_1/34/33/task-scheduler-demo-0.0.1-SNAPSHOT.jar
[INFO] 2024-03-30 12:31:32.735 +0800 - ****************************** Script Content ***************************************************************** [INFO] 2024-03-30 12:31:32.736 +0800 - Executing shell command : sudo -u root -i /data/bigdata/dolphinscheduler/exec/process/root/13044756119168/13099017794240_1/34/33/34_33.sh [INFO] 2024-03-30 12:31:32.740 +0800 - process start, process id is: 53661 [INFO] 2024-03-30 12:31:33.741 +0800 - -> ### Error: Unable to access jarfile /data/bigdata/dolphinscheduler/exec/process/root/13044756119168/13099017794240_1/34/33//data/bigdata/dolphinscheduler/exec/process/root/13044756119168/13099017794240_1/34/33/task-scheduler-demo-0.0.1-SNAPSHOT.jar [INFO] 2024-03-30 12:31:33.743 +0800 - process has exited. execute path:/data/bigdata/dolphinscheduler/exec/process/root/13044756119168/13099017794240_1/34/33, processId:53661 ,exitStatusCode:1 ,processWaitForStatus:true ,processExitValue:1 [INFO] 2024-03-30 12:31:33.743 +0800 - java task run result: TaskResponse(varPool=null, processId=53661, resultString=null, appIds=null, process=null, cancel=false, exitStatusCode=1, status=null) [INFO] 2024-03-30 12:31:33.744 +0800 - *********************************************************************************************** [INFO] 2024-03-30 12:31:33.744 +0800 - ********************************* Finalize task instance

[davidzollo]

@shuaihehe , as I have said in wechat, it's not very easy for you, you can have a try

[davidzollo]

@xmshdonghua , please paste the jar task info, I can't judge the problem through the log info

[davidzollo]

Hi guys, please in English, DS is a global open source project. You can goto https://asf-dolphinscheduler.slack.com and there is a users-china channel if you'd like to desc the question in chinese.

[asago-cis]

Hello - I installed the stand-alone from the latest binary here: https://www.apache.org/dyn/closer.lua/dolphinscheduler/3.2.1/apache-dolphinscheduler-3.2.1-bin.tar.gz and it defaulted to Chinese. Where is the configuration to use English? Is there an English version?

[dengd1937]

Hi @davidzollo , when i use the version 3.2.1 of dolphinscheduler execute python task, found the following error:

[INFO] 2024-04-17 13:24:29.220 +0800 - tenantCode :dolphinscheduler, task dir:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20
[INFO] 2024-04-17 13:24:29.220 +0800 - generate python script file:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20/py_6_20.py
[INFO] 2024-04-17 13:24:29.220 +0800 - #-*- encoding=utf8 -*-

print("This is a demo of python task")
[INFO] 2024-04-17 13:24:29.221 +0800 - Final Shell file is: 
[INFO] 2024-04-17 13:24:29.221 +0800 - ****************************** Script Content *****************************************************************
[INFO] 2024-04-17 13:24:29.221 +0800 - #!/bin/bash
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
${PYTHON_LAUNCHER} /tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20/py_6_20.py
[INFO] 2024-04-17 13:24:29.221 +0800 - ****************************** Script Content *****************************************************************
[INFO] 2024-04-17 13:24:29.221 +0800 - Executing shell command : sudo -u dolphinscheduler -i /tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20/6_20.sh
[INFO] 2024-04-17 13:24:29.228 +0800 - process start, process id is: 160775
[INFO] 2024-04-17 13:24:31.229 +0800 -  -> 
	-bash: /tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20/6_20.sh: Permission denied
[INFO] 2024-04-17 13:24:31.231 +0800 - process has exited. execute path:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20, processId:160775 ,exitStatusCode:126 ,processWaitForStatus:true ,processExitValue:126

But after i do some experiment on linux, give the result as follows:

So is this a bug?

[xxzuo]

Hello - I installed the stand-alone from the latest binary here: https://www.apache.org/dyn/closer.lua/dolphinscheduler/3.2.1/apache-dolphinscheduler-3.2.1-bin.tar.gz and it defaulted to Chinese. Where is the configuration to use English? Is there an English version?

hi, you can switch to the English version from here

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

[MrYaozhijian]

Hi @davidzollo , when i use the version 3.2.1 of dolphinscheduler execute python task, found the following error:

[INFO] 2024-04-17 13:24:29.220 +0800 - tenantCode :dolphinscheduler, task dir:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20
[INFO] 2024-04-17 13:24:29.220 +0800 - generate python script file:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20/py_6_20.py
[INFO] 2024-04-17 13:24:29.220 +0800 - #-*- encoding=utf8 -*-

print("This is a demo of python task")
[INFO] 2024-04-17 13:24:29.221 +0800 - Final Shell file is: 
[INFO] 2024-04-17 13:24:29.221 +0800 - ****************************** Script Content *****************************************************************
[INFO] 2024-04-17 13:24:29.221 +0800 - #!/bin/bash
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
${PYTHON_LAUNCHER} /tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20/py_6_20.py
[INFO] 2024-04-17 13:24:29.221 +0800 - ****************************** Script Content *****************************************************************
[INFO] 2024-04-17 13:24:29.221 +0800 - Executing shell command : sudo -u dolphinscheduler -i /tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20/6_20.sh
[INFO] 2024-04-17 13:24:29.228 +0800 - process start, process id is: 160775
[INFO] 2024-04-17 13:24:31.229 +0800 -  -> 
	-bash: /tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20/6_20.sh: Permission denied
[INFO] 2024-04-17 13:24:31.231 +0800 - process has exited. execute path:/tmp/dolphinscheduler/exec/process/dolphinscheduler/13298107086944/13298127307232_1/6/20, processId:160775 ,exitStatusCode:126 ,processWaitForStatus:true ,processExitValue:126

But after i do some experiment on linux, give the result as follows:

So is this a bug?

i use DS that version 3.2.1 also have the problem, friend ,are you solve it?