Increase allowed BOM size for uploads

[duanemay]

Current Behavior

When I upload a large sbom file using the PUT /v1/bom I get a 400 error with a message like:

String length (20051112) exceeds the maximum length (20000000) (through reference chain: org.dependencytrack.resources.v1.vo.BomSubmitRequest[\"bom\"]) (status: 400)

It appears this began in DependencyTrack 4.9.x

I have traced this to jackson-core's com.fasterxml.jackson.core.StreamReadContraints https://github.com/FasterXML/jackson-core/issues/1014

They have a mechanism here to change the limit. Please set it to a more reasonable default and/or provide a way to modify the limit via configuration.

Steps to Reproduce

1. Produce a large BOM with more than 20000000 characters.
2. Upload the SBOM via the PUT /v1/bom method.
3. See failure

Expected Behavior

BOM should upload

Dependency-Track Version

4.9.1

Dependency-Track Distribution

Executable WAR

Database Server

N/A

Database Server Version

N/A

Browser

Google Chrome

Checklist

• [X] I have read and understand the contributing guidelines
• [X] I have checked the existing issues for whether this defect was already reported

[z1atk0]

Hit by the same bug here. As a workaround, we are now running the syft generated .json through jq -c . before uploading it to the API server, which saves just enough unnecessary bytes to drop below the 20M limit.

[rkg-mm]

@nscuro running into the same issue in at least one project now which can't upload BOMs anymore. Can we put this in a bugfix release possibly?

[nscuro]

The problem here is that, when submitting the BOM in a single JSON request like this:

{
  "projectName": "acme-app",
  "projectVersion": "1.0.0",
  "bom": "<base64EncodedBom>"
}

the protection mechanism of Jackson kicks in. Some BOMs are huge, so even if we raise the limit, it will not solve the issue completely.

@rkg-mm Is it possible for you to adjust the upload request? There is another method that utilizes multipart/form-data requests instead, which will not have this limitation: https://docs.dependencytrack.org/usage/cicd/#cyclonedx

[rkg-mm]

@nscuro for the moment I found a quick fix by filtering unnecessary html code contained in some licenses that were automatically loaded from URLs into the BOM file, so for now I should be below the limit in all projects from what I know. However, this will surely be an issue at some point again

edit: but yes we could probably adjust our scripts to use the other endpoint, thanks for the hint edit2: maybe we could put this hint in an improved error message if this mechanism kicks in?

[esnible]

I modified my code to use multipart/form-data instead of https://github.com/DependencyTrack/client-go BOMService Upload().

If I want to upload 2 SBOMs to the same D-T Project, using BOMService Upload() the Project gets components from both SBOMs. Using multipart/form-data the Project gets only the components in the second uploaded SBOM. Adding a 5 second delay does not help.

[valentijnscholten]

@esnible DT only supports 1 SBOM per project+version, you can merge them upfront and upload them as 1 SBOM.

[nscuro]

Given there is already the POST /api/v1/bom endpoint that takes a multipart/form-data request, which does not suffer from this limitation, I don't believe there is value in raising the character limit for JSON payloads. Especially since it's a global setting, which we cannot apply to specific endpoints.

At the stage where the exception is thrown, we're already in a place where parsing and base64 decoding the offending value is causing unnecessary overhead and memory pressure anyway. The base64 encoding bloats the original BOM size, which negatively affects both the client, as well as the server with such large values.

So instead, I added an exception mapper that provides explicit hints when the condition is triggered for the PUT /api/v1/bom and PUT /api/v1/vex endpoints. I also updated the OpenAPI spec to point out this limitation, and suggest using the respective POST endpoints instead.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.