jib icon indicating copy to clipboard operation
jib copied to clipboard
verified publisher icon GoogleContainerTools

Maven jib:build fails if <outputPaths> contain non-existing directories

Open etzelc opened this issue 3 years ago • 3 comments

Environment:

  • Jib version: 3.3.0
  • Build tool: Maven
  • OS: Linux

Description of the issue: jib:build fails if you specify <outputPaths> which contain non-existing directories. For example the POM contains the following jib-maven-pluging configuration:

...
<!-- Jib -->
      <plugin>
        <groupId>com.google.cloud.tools</groupId>
        <artifactId>jib-maven-plugin</artifactId>
        <version>${jib-maven-plugin.version}</version>
        <configuration>
          <to>
            <image>gcr.io/REPLACE-WITH-YOUR-GCP-PROJECT/image-built-with-jib</image>
          </to>
          <outputPaths>
            <digest>${project.build.directory}/target/nonExistingDirectory/image.digest</digest>
            <imageId>${project.build.directory}/target/nonExistingDirectory/image.id</imageId>
            <imageJson>${project.build.directory}/target/nonExistingDirectory/image.json</imageJson>
        </outputPaths>
        </configuration>
        ...

It fails with [ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.3.0:build (default-cli) on project helloworld: /workspace/jib/examples/helloworld/target/target/nonExistingDirectory/image.digest

Expected behavior: jib:build creates directories specified in outputPaths and completes successful.

Steps to reproduce:

  1. use https://github.com/GoogleContainerTools/jib/tree/master/examples/helloworld
  2. Add <outputPaths> config from above and replace <to><image> accordingly
  3. Run mvn compile jib:build

Log output:

[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------------< example:helloworld >-------------------------
[INFO] Building helloworld 1
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ helloworld ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ helloworld ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- jib-maven-plugin:3.3.0:build (default-cli) @ helloworld ---
[INFO] 
[INFO] Containerizing application to registry.xyz...
[WARNING] Base image 'eclipse-temurin:8-jre' does not use a specific image digest - build may not be reproducible
[INFO] Using credentials from <to><auth> for registry.xyz
[INFO] The base image requires auth. Trying again for eclipse-temurin:8-jre...
[INFO] Using base image with digest: sha256:a3327e73ad173120e7a7cb21de45f4d50545efecc9969291c69a8095fa9d05c0
[INFO] 
[INFO] Container entrypoint set to [java, -cp, /app/resources:/app/classes:/app/libs/*, example.HelloWorld]
[INFO] 
[INFO] Built and pushed image as registry.xyz
[INFO] Executing tasks:
[INFO] [==============================] 100.0% complete
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  24.566 s
[INFO] Finished at: 2022-09-22T14:05:45Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.3.0:build (default-cli) on project helloworld: /workspace/jib/examples/helloworld/target/target/nonExistingDirectory/image.digest -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Additional information: Running mvn -e compile jib:build traces down the exception to JibBuildRunner.java:237. Here a Files.write(imageDigestOutputPath, imageDigest.getBytes(StandardCharsets.UTF_8)); is executed, which fails if a directory in the imageDigestOutputPath does not exist.

...
Caused by: java.nio.file.NoSuchFileException: /workspace/jib/examples/helloworld/target/target/nonExistingDirectory/image.digest
    at sun.nio.fs.UnixException.translateToIOException (UnixException.java:92)
    at sun.nio.fs.UnixException.rethrowAsIOException (UnixException.java:111)
    at sun.nio.fs.UnixException.rethrowAsIOException (UnixException.java:116)
    at sun.nio.fs.UnixFileSystemProvider.newByteChannel (UnixFileSystemProvider.java:219)
    at java.nio.file.spi.FileSystemProvider.newOutputStream (FileSystemProvider.java:478)
    at java.nio.file.Files.newOutputStream (Files.java:220)
    at java.nio.file.Files.write (Files.java:3425)
    at com.google.cloud.tools.jib.plugins.common.JibBuildRunner.runBuild (JibBuildRunner.java:237)
    at com.google.cloud.tools.jib.maven.BuildImageMojo.execute (BuildImageMojo.java:117)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)

etzelc avatar Sep 22 '22 14:09 etzelc

Currently outputPaths’s supported usage is either the default (project-dir)/target/jib-image or a custom-specified directory that is a valid (existing) path.

As a workaround (if this setup within target is needed by your project), you could also try using something like the maven-antrun-plugin to define mkdir tasks to set up the sub-directories within target as required.

emmileaf avatar Sep 22 '22 15:09 emmileaf

Thanks for the clarification and the workaround. I hadn't read it that way from the documentation.

I came across the issue because when you specify a custom directory for the tarball and run jib:buildTar, non-existing (sub)directories are created (presumably during the tarball creation?). That's why I expected the same for jib:build. For example run jib:buildTar with the following configuration and the build succeeds:

          <outputPaths>
            <tar>${project.build.directory}/nonExistingDirectory/image.tar</tar>
            <digest>${project.build.directory}/nonExistingDirectory/image.digest</digest>
            <imageId>${project.build.directory}/nonExistingDirectory/image.id</imageId>
            <imageJson>${project.build.directory}/nonExistingDirectory/image.json</imageJson>
        </outputPaths>

BTW: In the jib-maven-plugin/README.md is a minor typo in the text about 'Additional Build Artifacts'. It talks about <outputFiles> instead of <outputPaths>. https://github.com/GoogleContainerTools/jib/blob/eccebd0d52301fefce9017783c1c43fa8cc671c7/jib-maven-plugin/README.md?plain=1#L236

etzelc avatar Sep 22 '22 18:09 etzelc

I came across the issue because when you specify a custom directory for the tarball and run jib:buildTar, non-existing (sub)directories are created (presumably during the tarball creation?).

Ah, you are right - writing to tarball does have an additional step here to create parent directories if they do not exist. I don’t think this is currently supported for jib:build, but any PR contributions for either this enhancement or adding a note in the README would be welcomed.

BTW: In the jib-maven-plugin/README.md is a minor typo in the text about 'Additional Build Artifacts'. It talks about <outputFiles> instead of <outputPaths>.

Yes - thank you for catching this!

emmileaf avatar Sep 22 '22 20:09 emmileaf