AbsaOSS
Databricks- Support of Delta live tables
Background [Optional]
Hi I have a case of## Background [Optional]
Hi I have a case of using delta live tables that runs SQL notebooks I cannot add cell of python sc._jvm.za.co.absa.spline.harvester.SparkLineageInitializer.enableLineageTracking(spark._jsparkSession) since DLT doesnt support magic like %python So I created a seperate notebook of python - but still fails- although the spline Jar is installed on the cluster I think that becasue DLT manages dynamiclly the clusters - i think that all the installation of jars on the cluster take no effect
Question
Does someone has experiense using spline in databricks in DLT ? using delta live tables that runs SQL notebooks I cannot add cell of python sc._jvm.za.co.absa.spline.harvester.SparkLineageInitializer.enableLineageTracking(spark._jsparkSession) since DLT doesnt support magic like %python So I created a seperate notebook of python - but still fails- although the spline Jar is installed on the cluster I think that becasue DLT manages dynamiclly the clusters - i think that all the installation of jars on the cluster take no effect
Question
Does someone has experiense using spline in databricks in DLT ?
