spline-spark-agent icon indicating copy to clipboard operation
spline-spark-agent copied to clipboard
verified publisher icon AbsaOSS

use splineAgent as a Jar file

Open zacayd opened this issue 2 years ago • 12 comments

Hi i tried to install the Agent Spline without Maven - but Upload of the a Jar FIile i cloned the code from git and build it to a spline-spark-agent_2.12.jar and installed library using Upload to Databricks when i run the code

import scala.util.parsing.json.JSON
import za.co.absa.spline.harvester.SparkLineageInitializer._
import za.co.absa.spline.agent.AgentConfig
import za.co.absa.spline.harvester.postprocessing.AbstractPostProcessingFilter
import za.co.absa.spline.harvester.postprocessing.PostProcessingFilter
import org.apache.commons.configuration.Configuration
import za.co.absa.spline.harvester.conf.StandardSplineConfigurationStack
import za.co.absa.spline.harvester.HarvestingContext
import za.co.absa.spline.producer.model.ExecutionPlan
import za.co.absa.spline.producer.model.ExecutionEvent
import za.co.absa.spline.producer.model.ReadOperation
import za.co.absa.spline.producer.model.WriteOperation
import za.co.absa.spline.producer.model.DataOperation
import za.co.absa.spline.harvester.ExtraMetadataImplicits._
import za.co.absa.spline.harvester.SparkLineageInitializer._

I get error that it doesnt recognise the packages can you please advise what is missing?

zacayd avatar Oct 30 '23 15:10 zacayd

Hi, share the error, what jar are you uploading?

cerveada avatar Oct 31 '23 15:10 cerveada

HI i have attached the screen shot of the installation of the jar the jar is the artifact of build the project https://github.com/AbsaOSS/spline-spark-agent Screenshot 2023-11-01 130220

the errors are

command-4378807525288898:2: error: not found: value za import za.co.absa.spline.harvester.SparkLineageInitializer._ ^ command-4378807525288898:3: error: not found: value za import za.co.absa.spline.agent.AgentConfig ^ command-4378807525288898:4: error: not found: value za import za.co.absa.spline.harvester.postprocessing.AbstractPostProcessingFilter ^ command-4378807525288898:5: error: not found: value za import za.co.absa.spline.harvester.postprocessing.PostProcessingFilter ^ command-4378807525288898:6: error: object configuration is not a member of package org.apache.commons import org.apache.commons.configuration.Configuration ^ command-4378807525288898:7: error: not found: value za import za.co.absa.spline.harvester.conf.StandardSplineConfigurationStack ^ command-4378807525288898:8: error: not found: value za import za.co.absa.spline.harvester.HarvestingContext ^ command-4378807525288898:9: error: not found: value za import za.co.absa.spline.producer.model.ExecutionPlan ^ command-4378807525288898:10: error: not found: value za import za.co.absa.spline.producer.model.ExecutionEvent ^ command-4378807525288898:11: error: not found: value za import za.co.absa.spline.producer.model.ReadOperation ^ command-4378807525288898:12: error: not found: value za import za.co.absa.spline.producer.model.WriteOperation ^ command-4378807525288898:13: error: not found: value za import za.co.absa.spline.producer.model.DataOperation ^ command-4378807525288898:14: error: not found: value za import za.co.absa.spline.harvester.ExtraMetadataImplicits._ ^ command-4378807525288898:15: error: not found: value za import za.co.absa.spline.harvester.SparkLineageInitializer._ ^

zacayd avatar Nov 01 '23 11:11 zacayd

If you are modifying the code, you must build and use your own spark-3.3-spline-agent-bundle.jar not the one from maven.

cerveada avatar Nov 01 '23 13:11 cerveada

error: not found: value za

It looks like the Jar is either not seen by the Spark, or is empty/does not contain any Spline related classes. Please check your Jar content and the way you attach it to Spark.

wajda avatar Nov 01 '23 16:11 wajda

the jar is 5K - it was created when i made a build on the Intelij the file is called spline-spark-agent_2.12.jar i took the content of the POM.xml from Bundle 3.3 I then uploaded the jar as "Install New" on the Databricks Cluster Do you have a working solution of spline for Databricks which is not from Maven Central ?

zacayd avatar Nov 02 '23 11:11 zacayd

Please follow the building procedure from this section in the README - https://github.com/AbsaOSS/spline-spark-agent#building Then take the bundle jar from the bundle-3.3/target folder.

wajda avatar Nov 02 '23 15:11 wajda

i made this C:\Users\zacayd\Downloads\apache-maven-3.9.5-bin\apache-maven-3.9.5\bin>mvn –version

Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546)

Maven home: C:\Users\zacayd\Downloads\apache-maven-3.9.5-bin\apache-maven-3.9.5

Java version: 1.8.0_202, vendor: Oracle Corporation, runtime: C:\Program Files\Java\jre1.8.0_202

Default locale: en_US, platform encoding: Cp1252

OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

But Got

C:\Users\zacayd\Downloads\apache-maven-3.9.5-bin\apache-maven-3.9.5\bin>mvn scala-cross-build:change-version -Pscala-2.11

[INFO] Scanning for projects...

[WARNING] The requested profile "scala-2.11" could not be activated because it does not exist.

Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml

Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml

Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml (14 kB at 16 kB/s)

Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml (21 kB at 23 kB/s)

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 1.890 s

[INFO] Finished at: 2023-11-06T10:47:47+02:00

[INFO] ------------------------------------------------------------------------

[WARNING] The requested profile "scala-2.11" could not be activated because it does not exist.

[ERROR] No plugin found for prefix 'scala-cross-build' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (C:\Users\zacayd.m2\repository), central (https://repo.maven.apache.org/maven2)] -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/NoPluginFoundForPrefixException

From: Alex Vayda @.> Sent: Wednesday, 1 November 2023 18:08 To: AbsaOSS/spline-spark-agent @.> Cc: zacayd @.>; Author @.> Subject: Re: [AbsaOSS/spline-spark-agent] use splineAgent as a Jar file (Issue #761)

error: not found: value za

It looks like the Jar is either not seen by the Spark, or is empty/does not contain any Spline related classes. Please check your Jar content and the way you attach it to Spark.

— Reply to this email directly, view it on GitHub https://github.com/AbsaOSS/spline-spark-agent/issues/761#issuecomment-1789230582, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQCB5PB4R6PHNXSQ3A2PD53YCJXU7AVCNFSM6AAAAAA6WGALX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBZGIZTANJYGI . You are receiving this because you authored the thread.Message ID: < @.***>

zacayd avatar Nov 06 '23 08:11 zacayd

I've never seen this error. Something might be wrong with your POM files. Try it on a clean project. call the Maven command in the project root directory.

Here's how it should look like:

 ~/Projects/spline-spark-agent │ develop                                                                                                                                                      
❯ mvn scala-cross-build:change-version -Pscala-2.11
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] Spline Spark Agent                                                 [pom]
[INFO] agent-commons_2.12                                                 [jar]
[INFO] agent-core_2.12                                                    [jar]
[INFO] examples_2.12                                                      [jar]
[INFO] integration-tests_2.12                                             [jar]
[INFO] spark-2.2-spline-agent-bundle_2.11                                 [jar]
[INFO] spark-2.3-spline-agent-bundle_2.11                                 [jar]
[INFO] spark-2.4-spline-agent-bundle_2.12                                 [jar]
[INFO]
[INFO] -------< za.co.absa.spline.agent.spark:spline-spark-agent_2.12 >--------
[INFO] Building Spline Spark Agent 2.1.0-SNAPSHOT                         [1/8]
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spline-spark-agent_2.12 ---
[INFO]
[INFO] ----------< za.co.absa.spline.agent.spark:agent-commons_2.12 >----------
[INFO] Building agent-commons_2.12 2.1.0-SNAPSHOT                         [2/8]
[INFO]   from commons/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ agent-commons_2.12 ---
[INFO]
[INFO] -----------< za.co.absa.spline.agent.spark:agent-core_2.12 >------------
[INFO] Building agent-core_2.12 2.1.0-SNAPSHOT                            [3/8]
[INFO]   from core/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ agent-core_2.12 ---
[INFO]
[INFO] ------------< za.co.absa.spline.agent.spark:examples_2.12 >-------------
[INFO] Building examples_2.12 2.1.0-SNAPSHOT                              [4/8]
[INFO]   from examples/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ examples_2.12 ---
[INFO]
[INFO] --------< za.co.absa.spline.agent.spark:integration-tests_2.12 >--------
[INFO] Building integration-tests_2.12 2.1.0-SNAPSHOT                     [5/8]
[INFO]   from integration-tests/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ integration-tests_2.12 ---
[INFO]
[INFO] --< za.co.absa.spline.agent.spark:spark-2.2-spline-agent-bundle_2.11 >--
[INFO] Building spark-2.2-spline-agent-bundle_2.11 2.1.0-SNAPSHOT         [6/8]
[INFO]   from bundle-2.2/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.2-spline-agent-bundle_2.11 ---
[INFO]
[INFO] --< za.co.absa.spline.agent.spark:spark-2.3-spline-agent-bundle_2.11 >--
[INFO] Building spark-2.3-spline-agent-bundle_2.11 2.1.0-SNAPSHOT         [7/8]
[INFO]   from bundle-2.3/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.3-spline-agent-bundle_2.11 ---
[INFO]
[INFO] --< za.co.absa.spline.agent.spark:spark-2.4-spline-agent-bundle_2.12 >--
[INFO] Building spark-2.4-spline-agent-bundle_2.12 2.1.0-SNAPSHOT         [8/8]
[INFO]   from bundle-2.4/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.4-spline-agent-bundle_2.12 ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spline Spark Agent 2.1.0-SNAPSHOT:
[INFO]
[INFO] Spline Spark Agent ................................. SUCCESS [  0.078 s]
[INFO] agent-commons_2.12 ................................. SUCCESS [  0.009 s]
[INFO] agent-core_2.12 .................................... SUCCESS [  0.009 s]
[INFO] examples_2.12 ...................................... SUCCESS [  0.009 s]
[INFO] integration-tests_2.12 ............................. SUCCESS [  0.009 s]
[INFO] spark-2.2-spline-agent-bundle_2.11 ................. SUCCESS [  0.007 s]
[INFO] spark-2.3-spline-agent-bundle_2.11 ................. SUCCESS [  0.008 s]
[INFO] spark-2.4-spline-agent-bundle_2.12 ................. SUCCESS [  0.009 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.279 s
[INFO] Finished at: 2023-11-06T10:58:25+01:00
[INFO] ------------------------------------------------------------------------

wajda avatar Nov 06 '23 10:11 wajda

Hi

I successfully made mvn scala-cross-build:change-version -Pscala-2.11

mvn clean -Pscala-2.11,spark-2.4

also succeeded

but when in run

mvn install -Pscala-2.11,spark-2.4

I got

[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.5.0:compile (scala-compile) on project agent-commons_2.11: Execution scala-compile of goal net.alchim31.maven:scala-maven-plugin:4.5.0:compile failed.: CompileFailed -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.5.0:compile (scala-compile) on project agent-commons_2.11: Execution scala-compile of goal net.alchim31.maven:scala-maven-plugin:4.5.0:compile failed.

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2

(MojoExecutor.java:333)

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute

(MojoExecutor.java:316)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:212)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:174)

at org.apache.maven.lifecycle.internal.MojoExecutor.access$000

(MojoExecutor.java:75)

at org.apache.maven.lifecycle.internal.MojoExecutor$1.run

(MojoExecutor.java:162)

at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute

(DefaultMojosExecutionStrategy.java:39)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:159)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)

at

org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)

at org.apache.maven.lifecycle.internal.LifecycleStarter.execute

(LifecycleStarter.java:118)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)

at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)

at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)

at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)

at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)

at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke (Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke (Unknown Source)

at java.lang.reflect.Method.invoke (Unknown Source)

at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced

(Launcher.java:283)

at org.codehaus.plexus.classworlds.launcher.Launcher.launch

(Launcher.java:226)

at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode

(Launcher.java:407)

at org.codehaus.plexus.classworlds.launcher.Launcher.main

(Launcher.java:348)

Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile of goal net.alchim31.maven:scala-maven-plugin:4.5.0:compile failed.

at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo

(DefaultBuildPluginManager.java:133)

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2

(MojoExecutor.java:328)

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute

(MojoExecutor.java:316)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:212)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:174)

at org.apache.maven.lifecycle.internal.MojoExecutor.access$000

(MojoExecutor.java:75)

at org.apache.maven.lifecycle.internal.MojoExecutor$1.run

(MojoExecutor.java:162)

at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute

(DefaultMojosExecutionStrategy.java:39)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:159)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)

at

org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)

at org.apache.maven.lifecycle.internal.LifecycleStarter.execute

(LifecycleStarter.java:118)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)

at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)

at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)

at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)

at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)

at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke (Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke (Unknown Source)

at java.lang.reflect.Method.invoke (Unknown Source)

at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced

(Launcher.java:283)

at org.codehaus.plexus.classworlds.launcher.Launcher.launch

(Launcher.java:226)

at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode

(Launcher.java:407)

at org.codehaus.plexus.classworlds.launcher.Launcher.main

(Launcher.java:348)

Caused by: sbt.internal.inc.CompileFailed

at sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError

(IncrementalCompilerImpl.scala:335)

at sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally

(IncrementalCompilerImpl.scala:420)

at sbt.internal.inc.IncrementalCompilerImpl.compile

(IncrementalCompilerImpl.scala:137)

at sbt_inc.SbtIncrementalCompiler.compile

(SbtIncrementalCompiler.java:179)

at scala_maven.ScalaCompilerSupport.incrementalCompile

(ScalaCompilerSupport.java:356)

at scala_maven.ScalaCompilerSupport.compile

(ScalaCompilerSupport.java:113)

at scala_maven.ScalaCompilerSupport.doExecute

(ScalaCompilerSupport.java:88)

at scala_maven.ScalaMojoSupport.execute (ScalaMojoSupport.java:308)

at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo

(DefaultBuildPluginManager.java:126)

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2

(MojoExecutor.java:328)

at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute

(MojoExecutor.java:316)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:212)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:174)

at org.apache.maven.lifecycle.internal.MojoExecutor.access$000

(MojoExecutor.java:75)

at org.apache.maven.lifecycle.internal.MojoExecutor$1.run

(MojoExecutor.java:162)

at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute

(DefaultMojosExecutionStrategy.java:39)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute

(MojoExecutor.java:159)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)

at

org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)

at

org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)

at org.apache.maven.lifecycle.internal.LifecycleStarter.execute

(LifecycleStarter.java:118)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)

at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)

at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)

at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)

at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)

at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)

at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke (Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke (Unknown Source)

at java.lang.reflect.Method.invoke (Unknown Source)

at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced

(Launcher.java:283)

at org.codehaus.plexus.classworlds.launcher.Launcher.launch

(Launcher.java:226)

at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode

(Launcher.java:407)

at org.codehaus.plexus.classworlds.launcher.Launcher.main

(Launcher.java:348)

[ERROR]

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException

[ERROR]

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR] mvn -rf :agent-commons_2.11

From: Alex Vayda @.> Sent: Monday, 6 November 2023 12:04 To: AbsaOSS/spline-spark-agent @.> Cc: zacayd @.>; Author @.> Subject: Re: [AbsaOSS/spline-spark-agent] use splineAgent as a Jar file (Issue #761)

I've never seen this error. Something might be wrong with your POM files. Try it on a clean project. call the Maven command in the project root directory.

Here's how it should look like:

~/Projects/spline-spark-agent │ develop

❯ mvn scala-cross-build:change-version -Pscala-2.11

[INFO] Scanning for projects...

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Build Order:

[INFO]

[INFO] Spline Spark Agent [pom]

[INFO] agent-commons_2.12 [jar]

[INFO] agent-core_2.12 [jar]

[INFO] examples_2.12 [jar]

[INFO] integration-tests_2.12 [jar]

[INFO] spark-2.2-spline-agent-bundle_2.11 [jar]

[INFO] spark-2.3-spline-agent-bundle_2.11 [jar]

[INFO] spark-2.4-spline-agent-bundle_2.12 [jar]

[INFO]

[INFO] -------< za.co.absa.spline.agent.spark:spline-spark-agent_2.12 >--------

[INFO] Building Spline Spark Agent 2.1.0-SNAPSHOT [1/8]

[INFO] from pom.xml

[INFO] --------------------------------[ pom ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spline-spark-agent_2.12 ---

[INFO]

[INFO] ----------< za.co.absa.spline.agent.spark:agent-commons_2.12 >----------

[INFO] Building agent-commons_2.12 2.1.0-SNAPSHOT [2/8]

[INFO] from commons/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ agent-commons_2.12 ---

[INFO]

[INFO] -----------< za.co.absa.spline.agent.spark:agent-core_2.12 >------------

[INFO] Building agent-core_2.12 2.1.0-SNAPSHOT [3/8]

[INFO] from core/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ agent-core_2.12 ---

[INFO]

[INFO] ------------< za.co.absa.spline.agent.spark:examples_2.12 >-------------

[INFO] Building examples_2.12 2.1.0-SNAPSHOT [4/8]

[INFO] from examples/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ examples_2.12 ---

[INFO]

[INFO] --------< za.co.absa.spline.agent.spark:integration-tests_2.12 >--------

[INFO] Building integration-tests_2.12 2.1.0-SNAPSHOT [5/8]

[INFO] from integration-tests/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ integration-tests_2.12 ---

[INFO]

[INFO] --< za.co.absa.spline.agent.spark:spark-2.2-spline-agent-bundle_2.11 >--

[INFO] Building spark-2.2-spline-agent-bundle_2.11 2.1.0-SNAPSHOT [6/8]

[INFO] from bundle-2.2/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.2-spline-agent-bundle_2.11 ---

[INFO]

[INFO] --< za.co.absa.spline.agent.spark:spark-2.3-spline-agent-bundle_2.11 >--

[INFO] Building spark-2.3-spline-agent-bundle_2.11 2.1.0-SNAPSHOT [7/8]

[INFO] from bundle-2.3/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.3-spline-agent-bundle_2.11 ---

[INFO]

[INFO] --< za.co.absa.spline.agent.spark:spark-2.4-spline-agent-bundle_2.12 >--

[INFO] Building spark-2.4-spline-agent-bundle_2.12 2.1.0-SNAPSHOT [8/8]

[INFO] from bundle-2.4/pom.xml

[INFO] --------------------------------[ jar ]---------------------------------

[INFO]

[INFO] --- scala-cross-build:0.2.2:change-version (default-cli) @ spark-2.4-spline-agent-bundle_2.12 ---

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary for Spline Spark Agent 2.1.0-SNAPSHOT:

[INFO]

[INFO] Spline Spark Agent ................................. SUCCESS [ 0.078 s]

[INFO] agent-commons_2.12 ................................. SUCCESS [ 0.009 s]

[INFO] agent-core_2.12 .................................... SUCCESS [ 0.009 s]

[INFO] examples_2.12 ...................................... SUCCESS [ 0.009 s]

[INFO] integration-tests_2.12 ............................. SUCCESS [ 0.009 s]

[INFO] spark-2.2-spline-agent-bundle_2.11 ................. SUCCESS [ 0.007 s]

[INFO] spark-2.3-spline-agent-bundle_2.11 ................. SUCCESS [ 0.008 s]

[INFO] spark-2.4-spline-agent-bundle_2.12 ................. SUCCESS [ 0.009 s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 0.279 s

[INFO] Finished at: 2023-11-06T10:58:25+01:00

[INFO] ------------------------------------------------------------------------

— Reply to this email directly, view it on GitHub https://github.com/AbsaOSS/spline-spark-agent/issues/761#issuecomment-1794472867, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQCB5PGEQX7CIOJH2C274TTYDCY2NAVCNFSM6AAAAAA6WGALX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJUGQ3TEOBWG4 . You are receiving this because you authored the thread.Message ID: < @.***>

zacayd avatar Nov 06 '23 15:11 zacayd

There seems to be a compilation error. Are you compiling the original source code, or you changed something in it?

wajda avatar Nov 13 '23 18:11 wajda

the origanl code

zacayd avatar Nov 14 '23 08:11 zacayd

Sorry I cannot reproduce it. It builds fine on all our environments including CI and personal PCs. Please try to build it on another env. It must work.

wajda avatar Nov 14 '23 11:11 wajda