kyuubi
kyuubi copied to clipboard
[FEATURE] kyuubi server support to start spark client remotely
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Search before asking
- [X] I have searched in the issues and found no similar issues.
Describe the feature
At present, kyuubi server can only start the spark client under the same machine, and does not support starting the spark client under remote machines, which means that kyuubi server and spark client must be bound to the same machine, but the actual production environment needs to be deployed separately, whether to consider support to start spark client remotely.
Motivation
No response
Describe the solution
No response
Additional context
No response
Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
I am not sure whether it is a generic requirement.
Seems that you can wrap the spark-submit script under $SPARK_HOME/bin to implement it.
wrap the spark-submit with
ssh $REMOTE_HOST "$REMOTE_SPARK_HOME/bin/spark-submit $@"