azuredevops dependency-check-build-task@6 fails on windows-latest hosted agent since release of DependencyCheck 11.0.0

Describe the bug My pipeline was working as expected this morning, but is not failing with the error shown below, I had made YAML edits

C:\Windows\system32\cmd.exe /D /S /C "D:\a\_tasks\dependency-check-build-task_47ea1f4a-57ba-414a-b12e-c44f42765e72\6.2.3\dependency-check\bin\dependency-check.bat --version"
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError: org/owasp/dependencycheck/App has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:621)

Looking at the DependencyCheck project site it seems version 11.0.0 was shipped around 11am this morning, around the time the error started to appear.

It seems 11.0.0 requires a newer version of JAVA than 10.x.x required, and though the windows-latest hosted build agents have various version of JAVA installed, the task is defaulting to use JDK 8 (52.0) To Reproduce Steps to reproduce the behavior:

Use the task as documented in this repo

Expected behavior Analysis should occur when the current V11.x.x of the underlying tool is automatically downloaded used.

Oct 21 '24 12:10 rfennell

I have a workaround for this issue, explicably set the JAVA version using the JavaToolInstaller@0 task. As I am using the Microsoft hosted build agent, with multiple JAVA versions installed, all it needs to do is update selected version, but the same task can be used to install a JAVA version if it is missing on a self hosted agent.

This is done using the following YAML

  - task: JavaToolInstaller@0
    inputs:
      versionSpec: '11'
      jdkArchitectureOption: 'x64'
      jdkSourceOption: 'PreInstalled'
      
  - task: dependency-check-build-task@6
    displayName: Dependency Check
    inputs:
      projectName: 'My Project'
      scanPath: 'src'
      format: 'HTML,XML'
      additionalArguments: '--nvdApiKey $(nvdapikey)'

I think this is perfectly valid solution, but I wanted to log it as an issue so either

Update the documentation, happy to provide a PR if you tell me where it should go
Update the JAVA_HOME settings inside the tasks itself, but this could have knock on effects.

Oct 21 '24 12:10 rfennell

The joys of using latest - even when latest could have bumped a major version indicating breaking changes... semantic versioning ftw.

Oct 21 '24 12:10 jeremylong

It gets better...

To speed up the running of this task we use a pipeline cache to pre-populate the vulnerabilities DB to save up to 15 min on a pipeline run

      - task: Cache@2
         inputs:
            key: '"NVDCache" | "$(Agent.OS)"'
            restoreKeys: |
              NVDCache | "$(Agent.OS)"
              NVDCache
            path: $(nvdcachepath)
          displayName: NVD Cache

It seems with 11.0.0 the vulnerabilities DB format has changed.

ERROR] Incompatible or corrupt database found. To resolve this issue please remove the existing database by running purge
[ERROR] One or more fatal errors occurred
[ERROR] Unable to connect to the dependency-check database

The fix is simple, alter the key name of the cache so a new one is created. This is easier than trying to purge the old copy for any given branch. The cache is short lived and will be deleted in a 7 days, so there will be old data left around

      - task: Cache@2
         inputs:
            key: '"NVDCache1" | "$(Agent.OS)"'
            restoreKeys: |
              NVDCache1 | "$(Agent.OS)"
              NVDCache1
            path: $(nvdcachepath)
          displayName: NVD Cache

But another change to be aware of

Oct 21 '24 13:10 rfennell

I still get error for

- task: JavaToolInstaller@0
  inputs:
    versionSpec: '11'
    jdkArchitectureOption: 'x64'
    jdkSourceOption: 'PreInstalled'

- task: dependency-check-build-task@6
  inputs:
    projectName: 'DevSecOps'
    scanPath: '**/*.js'
    enableVerbose: true
    additionalArguments: '--updateonly --data "$(Build.ArtifactStagingDirectory)/owaspdata"'
    nvdApiKey: '$(NVDApiKey)'
  displayName: OWASP Data Update

Error :

[INFO] Downloaded 230,000/266,660 (86%)
[INFO] Downloaded 240,000/266,660 (90%)
[ERROR] Error updating the NVD Data
org.owasp.dependencycheck.data.update.exception.UpdateException: Error updating the NVD Data
	at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi(NvdApiDataSource.java:397)
	at org.owasp.dependencycheck.data.update.NvdApiDataSource.update(NvdApiDataSource.java:117)
....

Caused by: io.github.jeremylong.openvulnerability.client.nvd.NvdApiException: Failed to parse NVD data
	at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient._next(NvdCveClient.java:363)
	at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next(NvdCveClient.java:331)
	at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi(NvdApiDataSource.java:353)
	... 6 common frames omitted
Caused by: com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of `io.github.jeremylong.openvulnerability.client.nvd.CvssV4Data$ExploitMaturityType`, problem: PROOF-OF-CONCEPT
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 803450] (through reference chain: io.github.jeremylong.openvulnerability.client.nvd.CveApiJson20["vulnerabilities"]->java.util.ArrayList[357]->io.github.jeremylong.openvulnerability.client.nvd.DefCveItem["cve"]->io.github.jeremylong.openvulnerability.client.nvd.CveItem["metrics"]->io.github.jeremylong.openvulnerability.client.nvd.Metrics["cvssMetricV40"]->java.util.ArrayList[0]->io.github.jeremylong.openvulnerability.client.nvd.CvssV4["cvssData"]->io.github.jeremylong.openvulnerability.client.nvd.CvssV4Data["exploitMaturity"])
	at com.fasterxml.jackson.databind.exc.ValueInstantiationException.from(ValueInstantiationException.java:47)
...

[INFO] Updating CISA Known Exploited Vulnerability list: https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json
[INFO] Begin database defrag
[INFO] End database defrag (5893 ms)
[ERROR] Error updating the NVD Data
org.owasp.dependencycheck.data.update.exception.UpdateException: Error updating the NVD Data

...

Dependency Check completed with exit code 8.
Dependency Check reports:
[]
Dependency Check failed with message "Dependency Check exited with an error code (exit code: 8)."
##[error]Dependency Check exited with an error code (exit code: 8).

Oct 21 '24 21:10 ryan-payet-wcc

The java.lang.IllegalArgumentException: PROOF-OF-CONCEPT error is being tracked by jeremylong/DependencyCheck/issues/7067. The supposed root-cause is explained quite nicely in this comment.

Oct 21 '24 21:10 phunkeler

Can this task be updated to find an appropriate Java version automatically? So we don't have to change the default in every pipeline from 8 to 11? We run pipelines on the Microsoft image windows-2022 which comes with Java versions 8 (default), 11, 17, and 21, and each has an environment variable to find it as described here.

Oct 23 '24 02:10 simon-biber

I think the issues is this task downloads the 'current' version of the DependancyCheck tool. It does not know about any change in requirement for a new release of the tool, so automation of the correct version is non-trivial

Oct 23 '24 10:10 rfennell

it was quite easy to fix for us:

We added a step with --purge instead of update database.

Store the database as pipeline artifact and usually just updates it.

Added the purge step once, ran the pipeline and then disabled the purge step and everything just works fine. In our setup we have one pipeline that updates the database. and other pipelines that uses that pipelines artifact (cve database) to run the dependency checks. So the pipeline below is what we have usually (Except for the step "Dependency Check purge DB -- only used when 11. release broke stuff")

pool: name: Azure Pipelines

steps:

powershell: | New-Item -Path "$(Pipeline.Workspace)" -Name "cvedata" -ItemType "directory"

displayName: 'PowerShell Script'
task: DownloadPipelineArtifact@2 displayName: 'Download Pipeline Artifact' inputs: buildType: specific project: '26c15e85-e733-4cfb-94fc-e82d9a7fa137' definition: 838 allowPartiallySucceededBuilds: true artifactName: CVEDatabase targetPath: '$(Pipeline.Workspace)\cvedata' continueOnError: true
task: JavaToolInstaller@0 displayName: 'Use Java 11' inputs: versionSpec: 11 jdkArchitectureOption: x64 jdkSourceOption: PreInstalled
task: dependency-check.dependencycheck.dependency-check-build-task.dependency-check-build-task@6 displayName: 'Dependency Check purge DB -- only used when 11. release broke stuff' inputs: projectName: DBUpdate scanPath: '$(Build.SourcesDirectory)' format: ALL reportsDirectory: '$(Agent.TempDirectory)/dependency-scan-results' additionalArguments: '--purge --data "$(Pipeline.Workspace)\cvedata"'
task: dependency-check.dependencycheck.dependency-check-build-task.dependency-check-build-task@6 displayName: 'Dependency Check' inputs: projectName: DBUpdate scanPath: '$(Build.SourcesDirectory)' format: ALL reportsDirectory: '$(Agent.TempDirectory)/dependency-scan-results' additionalArguments: '--updateonly --data "$(Pipeline.Workspace)\cvedata"'
task: PublishPipelineArtifact@1 displayName: 'Publish Pipeline Artifact' inputs: targetPath: '$(Pipeline.Workspace)\cvedata' artifact: CVEDatabase

And this is another pipeline that uses that pipelineartifact to run checks on a repo

pool: name: Azure Pipelines

steps:

task: DownloadPipelineArtifact@2 displayName: 'Download Pipeline Artifact' inputs: buildType: specific project: '26c15e85-e733-4cfb-94fc-e82d9a7fa137' definition: 838 artifactName: CVEDatabase targetPath: '$(Pipeline.Workspace)\cvedata'
task: JavaToolInstaller@0 displayName: 'Use Java 11' inputs: versionSpec: 11 jdkArchitectureOption: x64 jdkSourceOption: PreInstalled
task: dependency-check.dependencycheck.dependency-check-build-task.dependency-check-build-task@6 displayName: 'Dependency Check' inputs: projectName: ApsisAdapter scanPath: '**/*.csproj' format: JUNIT failOnCVSS: 0 warnOnCVSSViolation: true additionalArguments: '--noupdate --data "$(Pipeline.Workspace)\cvedata"'
task: PublishTestResults@2 displayName: 'Publish OWASP Results' inputs: testResultsFiles: 'dependency-check*junit.xml' searchFolder: '$(Common.TestResultsDirectory)' failTaskOnFailedTests: true testRunTitle: 'Dependency Check'

Oct 23 '24 11:10 Xezol

I think it's out of scope for this task to install a fitting Java version, that's for another step in the pipeline to do.

Regrading using the latest version that's a bit dangerous when there's breaking changes, but it's actually a setting for this task - maybe this is solved by making this setting a bit more explicit?

Nov 06 '24 07:11 Saturate

I agree with Saturate from above.

In my situation speed of getting back to deployment was more critical, so I took the path of pinning the dependency check CLI to the last working version, v10.0.4. See below for the line I added to the pipeline task inputs list:

task: dependency-check-build-task@6
    condition: succeeded()
    displayName: Run Checks
    inputs:
      additionalArguments: >
        --data $(nvdCacheDirectory)
        --noupdate
      dependencyCheckVersion: 10.0.4    <-- ADDED THIS LINE
      failOnCVSS: $(FailOnCVSS)
      format: HTML, JSON, JUNIT
      scanPath: '**/*.csproj

My pipeline was back up and running without issue. When I have more time I will dig into making the correct Java version available and purging the older NVD cache data.

Nov 08 '24 18:11 TheHokieCoder