dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

[Bug] [Flink、Spark] Tasks such as flink and spark fail to run after kerberos authentication is enabled on Dolphinscheduler

Open gaotong521 opened this issue 1 year ago • 1 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

When kerberos authentication is enabled, the kerberos expiration time is 7 days, and tasks such as flink and spark will fail to be run after 7 days. The kerberos code logic for periodically updating has been added and is valid, allowing the resource center to use it properly. However, the hand running flink and spark tasks still fails and the Dolphinscheduler cluster needs to be restarted

The newly added code is shown below: image image

After 7 days, the resource center can be used normally, as shown in the following figure image

Running the flink live spark task will fail after 7 days, as shown below:

image image image

What you expected to happen

The flink and spark tasks can run properly at any time without failing when kerberos expires, and the dolphinscheduler cluster does not need to be restarted

How to reproduce

The expiration time of kerberos is 7 days, after which running tasks will fail

Anything else

no

Version

3.2.x

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

gaotong521 avatar Nov 25 '24 09:11 gaotong521

After renew the tgt, you need to reconnect to HDFS, eg. https://github.com/apache/dolphinscheduler/pull/17394

wuzhenhua01 avatar Aug 05 '25 14:08 wuzhenhua01