sparklens icon indicating copy to clipboard operation
sparklens copied to clipboard
verified publisher icon qubole

support in spark 3.x version

Open abdul2011 opened this issue 5 years ago • 1 comments

Does sparklens supported with spark 3.x version

abdul2011 avatar Mar 30 '21 11:03 abdul2011

i need to know this as well. i tried running a simple WordCount application with spark-3.2.1 and kubernetes-1.20 cluster setup. i added the lines --jars /path/to/sparklens-0.3.2-s_2.11.jar --conf spark.extraListeners=com.qubole.sparklens.QuboleJobListener in my spark-submit query. the result was that the executor pods keep on running INFINITELY. they never stop and always stay in "Running" status.

link to kubectl logs

Update: this bug seems to happen with Spark-Pi example as well :\

utkarsh92 avatar May 18 '22 09:05 utkarsh92