byte-buddy icon indicating copy to clipboard operation
byte-buddy copied to clipboard

Allow ByteBuddy plugins to be detected from the application's classpath

Open odrotbohm opened this issue 3 years ago • 25 comments

ByteBuddy plugins currently have to be declared as dependencies of the ByteBuddy plugin itself. Unfortunately, those dependencies are not subject to versions defined in a <dependencyManagement /> section. BOM POMs have become a means of defining sets of library versions that are known to work with each other. The explicit plugin declaration requirement escapes such a definition and causes an additional version having to be defined, which subverts the purpose of the BOM.

I wonder if plugins could also be found on the compile classpath of the project. That would allow the plugin dependencies to be defines in provided scope so that they do not leak into the runtime classpath but are still available via compilation.

odrotbohm avatar Jul 20 '22 07:07 odrotbohm

I think I would rather try to plug into the dependencyManagement configuration and try to extract that information for the case that the version is not specified. I'll see how I can leverage this

raphw avatar Jul 21 '22 17:07 raphw

I implemented this on master, if you wanted to give it a try.

raphw avatar Jul 22 '22 10:07 raphw

That unfortunately doesn't seem to help as Maven rejects the dependency not having a version declared in the first place. 😕

odrotbohm avatar Jul 22 '22 10:07 odrotbohm

Bummer, ideally, this would be fixed by Maven but I'll see to add such an option.

raphw avatar Jul 23 '22 18:07 raphw

As far as I understand, build plugins run with the dedicate set of library versions that have been defined for them. Considering the <dependencyManagement /> section for those might cause them to fail, as the user dependency might alter the plugin dependency versions incompatibly.

I guess the user-provided plugins for the ByteBuddy plugin are a slightly different beast though, as they're much more align with the actual project code's dependencies. I guess we could discuss this with the Maven team, but I guess this case here is a bit of a special one that they're not willing to change their defaults for.

odrotbohm avatar Jul 24 '22 09:07 odrotbohm

I added the option on master now. I see how it can be useful, but I also see how one can shoot oneself in the foot. As with previous philosophy, I leave the responsibility of the options to the user.

raphw avatar Jul 25 '22 20:07 raphw

I just gave this a spin and see the following exception:

Caused by: java.net.MalformedURLException: no protocol: /Users/odrotbohm/Documents/workspace/maws/acme-commerce/target/classes
    at java.net.URL.<init> (URL.java:674)
    at java.net.URL.<init> (URL.java:569)
    at java.net.URL.<init> (URL.java:516)
    at net.bytebuddy.build.maven.ByteBuddyMojo$Transformer$ForDiscoveredPlugin$FromClassLoader.toClassLoader (ByteBuddyMojo.java:927)
    at net.bytebuddy.build.maven.ByteBuddyMojo.apply (ByteBuddyMojo.java:414)
    at net.bytebuddy.build.maven.ByteBuddyMojo.execute (ByteBuddyMojo.java:323)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:568)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)

Steps to reproduce:

$ git clone https://github.com/odrotbohm/mawspring
$ cd mawspring/acme-commerce
$ ./mvnw clean test-compile -e

Is there a file: prefix missing?

odrotbohm avatar Jul 25 '22 21:07 odrotbohm

I lacked a test still. URL needs a file prefix. Will fix tomorrow.

raphw avatar Jul 25 '22 21:07 raphw

Alright, better to construct a File and let the VM do the conversion. I am still messing with my build, I will try a release once it is stable again.

raphw avatar Jul 26 '22 09:07 raphw

I am running into classloading issues now:

Caused by: org.apache.maven.plugin.MojoExecutionException: Cannot resolve plugin: org.jmolecules.bytebuddy.JMoleculesPlugin
    at net.bytebuddy.build.maven.ByteBuddyMojo.apply (ByteBuddyMojo.java:417)
    at net.bytebuddy.build.maven.ByteBuddyMojo.execute (ByteBuddyMojo.java:323)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:568)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:568)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:53)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:152)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:76)
Caused by: java.lang.NoClassDefFoundError: net/bytebuddy/build/Plugin
    at java.lang.ClassLoader.defineClass1 (Native Method)
    at java.lang.ClassLoader.defineClass (ClassLoader.java:1012)
    at java.security.SecureClassLoader.defineClass (SecureClassLoader.java:150)
    at java.net.URLClassLoader.defineClass (URLClassLoader.java:524)
    at java.net.URLClassLoader$1.run (URLClassLoader.java:427)
    at java.net.URLClassLoader$1.run (URLClassLoader.java:421)
    at java.security.AccessController.doPrivileged (AccessController.java:712)
    at java.net.URLClassLoader.findClass (URLClassLoader.java:420)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:587)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:520)
    at java.lang.ClassLoader.defineClass1 (Native Method)
    at java.lang.ClassLoader.defineClass (ClassLoader.java:1012)
    at java.security.SecureClassLoader.defineClass (SecureClassLoader.java:150)
    at java.net.URLClassLoader.defineClass (URLClassLoader.java:524)
    at java.net.URLClassLoader$1.run (URLClassLoader.java:427)
    at java.net.URLClassLoader$1.run (URLClassLoader.java:421)
    at java.security.AccessController.doPrivileged (AccessController.java:712)
    at java.net.URLClassLoader.findClass (URLClassLoader.java:420)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:587)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:520)
    at java.lang.Class.forName0 (Native Method)
    at java.lang.Class.forName (Class.java:467)
    at net.bytebuddy.build.maven.ByteBuddyMojo.apply (ByteBuddyMojo.java:414)

I guess that's due to my plugin class being picked up from the project classpath it's also tried to be loaded against that one?

odrotbohm avatar Jul 26 '22 10:07 odrotbohm

Yes, indeed, I assume that the Byte Buddy plugin is not on the right class path. Likely, I need to resolve the Maven plugin's class loader as a parent.

raphw avatar Jul 26 '22 10:07 raphw

Try now!

raphw avatar Jul 26 '22 10:07 raphw

Boom, works! 🥳

odrotbohm avatar Jul 26 '22 11:07 odrotbohm

In the process of declaring less non-optional dependencies on my plugin (to avoid all of them leak into the user compile space if they use the plugin as provided dependency) I am running into an issue: in the current mode of operation (plugin declared explicitly as ByteBuddy plugin dependency) as the dependencies are now declared as optional, they're not available on the plugin's classpath out of the box when instantiating types. I have the instantiations guarded by inspection of the ClassFileLocator for the presence of a certain class. Judging from the stack trace, that locator "sees" types available on the user classpath, but the subsequent object instantiation that refers to types from the user classpath, which are now optional dependencies of the plugin does not happen on a classpath including the user project's dependencies.

Is there something obvious we can do about this? The way it looks to me right now, I'll have to decide for one of the two approaches: my ByteBuddy plugin using non-optional dependencies to work in the explicit declaration scenario or switching to optional dependencies and thus "breaking" the old way (unless users declare the transitive dependencies, too).

odrotbohm avatar Jul 26 '22 12:07 odrotbohm

I guess I'm gonna provide a dedicated artifact with all non-optional dependencies removed for usage with the user classpath plugin detection. Thanks for your assistance. Feel free to close as complete.

odrotbohm avatar Jul 26 '22 13:07 odrotbohm

I was looking through Maven's APIs (the 10.000 that it offers) and I cannot really see how I can adjust it to give me optional dependencies or not. I only receive a class path as a list of files. If you have a better solution, please let me know!

raphw avatar Jul 26 '22 13:07 raphw

I see that there is a getArtifacts method in the Maven API where such filter elements are available. I will see how I can refactor Byte Buddy to include some filters such as including optional or runtime scoped types by properties to make this a bit more convenient to use.

raphw avatar Jul 26 '22 22:07 raphw

Can you try the with-optional-config branch and possibly play a bit with it? I think this should make it possible to solve your issue. I am however not yet sure what the default should be.

raphw avatar Jul 26 '22 22:07 raphw

I've switched to the additional artifact for my Plugin implementation. I guess it's quite a special one as it entirely reacts to the user classpath arrangement and thus has a pretty broad compile time dependency arrangement that doesn't make it a great fit for the use as provided dependency. The additional non-dependency artifact works fine with me, so I could totally understand if you don't want to drive up the complexity of all this any further.

odrotbohm avatar Jul 27 '22 09:07 odrotbohm

It's just a flag and a few lines of codes to add the optional support, so if it helps you, I am happy to add it. Does the branch solve your problem?

raphw avatar Jul 27 '22 10:07 raphw

I just gave this a try and I can see the classpath elements being added if I set the flag. That said, what I am stumbling above is not the types not visible during the transformation (via ClassFileLocator) but the actual target plugin type not being able to be instantiated as the ByteBuddy Maven plugin execution classpath doesn't seem to see types of optional dependencies. I am uncertain if we can actually change something about this from our side, or whether that'd require a change in Maven.

All that said, I am just fine with a dedicated artifact for that special mode of operation. Not needing to declare a version for a build plugin dependency as shown here helps a lot.

odrotbohm avatar Jul 28 '22 07:07 odrotbohm

Strangely enough, I am using the same sources for this discovery as I use for applying plugins. Do you have a simple reproducer? I would give mvnDebug a try then.

raphw avatar Jul 28 '22 11:07 raphw

This commit is trying to optionalize some dependencies of the plugin.

$ git clone https://github.com/xmolecules/jmolecules-integrations
$ cd jmolecules-integrations/jmolecules-bytebuddy
$ git checkout hacking/optional-dependencies
$ mvn clean test

should reproduce the failure.

odrotbohm avatar Jul 28 '22 12:07 odrotbohm

I gave it another try stepping through but it seems like this is built in in Maven's artifact resolver and cannot be changed.

raphw avatar Jul 28 '22 13:07 raphw

Never mind, thanks for the extensive support!

odrotbohm avatar Jul 28 '22 14:07 odrotbohm