Finalize the implementation of the Sonatype strategy by implementing API payload and rate limits

[antoine-coulon]

For now, the strategy around Sonatype is not taking into account rate and payload limits imposed by the API.

• Payload limit

When requesting multiple components, there is a payload limit of 128 components per requests). Given that a project might contains thousands of dependencies, we must provide a way to batch requests respecting taking care of that limit.

Issue: For now, all the dependencies found from the scanner are provided in the body of the request here . This is problematic because when dealing with a lot of dependencies, there is at the moment only one http request fired with all the dependencies, whereas we should batch X requests which each containing a maximum of 128 dependencies by payload. When providing more than 128 components per request, the API responds with an error.

Goal: Take for instance an array of package urls with 130 components (130 dependencies formatted as package urls). The goal is the create from this array chunks of size 128 and to fire an HTTP request to the API with each chunk. In this example we would have:

• 1 HTTP request with components from 1 to 128

• 1 HTTP request with components from 128 to 130

• API rate limit

Probably needs an implementation using @myunisoft/httpie.

Issue: Nothing specific with vuln here, but the interactions with the Sonatype API should handle rate limit through the use of the @myunisoft/httpie http client. Without throttling requests, we are sure to exceed the rate limit of the API.

Goal: Use @myunisoft/httpie to configure rate limit, there is an example here