[Feature][PyDolphinScheduler] Support more task plugin in PyDolphinScheduler

[jieguangzhou]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Description

DolphinScheduler has recently added a number of task plugins. We have to add more support in pydolphinscheduler. List of types in DolphinScheduler: https://dolphinscheduler.apache.org/en-us/docs/dev/user_doc/guide/task/java.html List of types in PydolphinScheduler: https://dolphinscheduler.apache.org/python/dev/tasks/index.html

There are task types in DolphinScheduler but not in PydolphinScheduler:

• [ ] Pigeon
• [ ] SeaTunnel
• [ ] Amazon EMR
• [ ] Apache Zepplin #11908
• [ ] Jupyter #11907
• [ ] Hive CLI #11909
• [ ] Kubernetes
• [x] MLflow #11923
• [x] OpenMLDB #11906
• [x] DVC #11922
• [ ] Dinky #11916
• [ ] Java #11914
• [x] SageMaker #11921
• [ ] ChunJun #11915
• [x] Pytorch #11924

After finished the pydolphinscheduler implementation, Hope to supplement the YAML sample, if possible

Like this: https://dolphinscheduler.apache.org/python/dev/tutorial.html#overview-of-tutorial

https://github.com/apache/dolphinscheduler/tree/dev/dolphinscheduler-python/pydolphinscheduler/examples/yaml_define

Use case

No response

Related issues

No response

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[jieguangzhou]

@zhongjiajie @SbloodyS @EricGao888 Can we add the tag of Good First Issue?

[SbloodyS]

@zhongjiajie @SbloodyS @EricGao888 Can we add the tag of Good First Issue?

Done.

[github-actions[bot]]

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.

• In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
• If you haven't received a reply for a long time, you can join our slack and send your question to channel #troubleshooting

[EricGao888]

May I pick those plugins I contributed before? Jupyter, HiveCLI, Zeppelin 🤣 I could handle the corresponding tasks in pyds : )

• [ ] jupyter #11907
• [ ] zeppelin #11908
• [ ] hive cli #11909

[jieguangzhou]

May I pick those plugins I contributed before? Jupyter, HiveCLI, Zeppelin 🤣 I could handle the corresponding tasks in pyds : )

Of course.

[jieguangzhou]

I want to pick these plugins

• [x] MLflow #11923
• [x] DVC #11922
• [x] SageMaker #11921
• [x] Pytorch #11924
• [x] OpenMLDB #11944

If others are interested in contributing, they can comment on this issue

[DarkAssassinator]

May i pick chunjun, Dinky and Java?

• [ ] JAVA #11914
• [ ] chunjun #11915
• [ ] Dlinky #11916

[EricGao888]

May i pick chunjun, Dinky and Java?

@DarkAssassinator @jieguangzhou Sure, thanks for the help! Could u plz create related sub-issues like this comment?https://github.com/apache/dolphinscheduler/issues/11906#issuecomment-1245011210 Thanks : )