Runbook for dependency debugging featuring Linkage Checker

[lukecwik]

It would be great if there was a runbook which we could point users of Apache Beam and also Google Cloud customers when resolving linkage errors between their project and Apache Beam/Google Cloud libraries.

[suztomo]

FYI, Google-cloud-java team has https://github.com/googleapis/google-cloud-java/blob/master/TROUBLESHOOTING.md#classnotfoundexception-nosuchmethoderror-noclassdeffounderror

If you can't modify and push new versions of your dependencies, import com.google.cloud:libraries-bom:3.4.0 (or a more recent version) and use that to select consistent dependency versions.

Yet, we observed cases where libraries-bom cannot solve NoClassDefFoundErrors or NoSuchMethodErrors with Apache Beam. That's why I'm contributing to the dependency upgrades.

[suztomo]

Memo: Understand problem:

• identify caller
• identify callee
• get dependency tree node for them.

Fix?

Runbook Draft

1. Maven artifacts in GCP Libraries BOM

Let the GCP Libraries BOM specify the versions of your dependencies for Maven artifacts listed in https://storage.googleapis.com/cloud-opensource-java-dashboard/com.google.cloud/libraries-bom/snapshot/artifact_details.html (replace "snapshot" with your BOM version, such as 4.0.1).

Maven

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>libraries-bom</artifactId>
        <version>4.0.1</version>
        <type>pom</type>
        <scope>import</scope>
       </dependency>
     </dependencies>
  </dependencyManagement>

Gradle

dependencies {
    api platform('com.google.cloud:libraries-bom:4.0.1')
}

For more details, refer to Gradle: The Java Platform Plugin.

2. Other Dependencies Outside GCP Libraries BOM

Use the version in "Recommended Versions" section of Google Cloud Platform Java Dependency Dashboard (replace the "snapshot" in the URL with your GCP Libraries BOM versoin).

3. Libraries Outside the Dashboard

For libraries still not listed in the previous section, use https://search.maven.org/ to find the latest version of library.

How to read "v1-rev20200131-1.30.8" in Google Libraries

Some Google libraries have version number that consists of several components. For example,com.google.apis:google-api-services-iam's versions include "v1-rev20200131-1.30.8", "v1-rev20191213-1.29.2", "v1-rev20191213-1.28.0". This version scheme consists of 3 components separated by dashes: server-side API version, release date, and Google API client version.

In the example of "v1-rev20200131-1.30.8",

• "v1" is the server -side API version
• "rev20200131" is the release date, and
• "1.30.8" is Google API client version.

When you encounter such versions, select the version with "1.30.X' in Google API client version and the latest release date. There should be one server-side API versions after this criteria.