Twistlock API calls do not consider pagination and are limited to the first 50 results

Open kteague-tasktop opened this issue 4 years ago • 1 comments

The Twistlock integration does not handle paginated results from the Twistlock API. The Twistlock API is documented to only return the first 50 results from any given API call:

https://prisma.pan.dev/api/cloud/cwpp/21-08/api-limits/

We encountered this issue when using the datadog-agent (7.32.4, latest) to pull twistlock metrics and ingest into Datadog. We have an environment with 159 hosts and each host has 1 CVE, so expected to see a result of 159, but only saw 50 (we also have environments with 6 and 48 hosts and the results there were as expected).

We traced the call to get the /api/v1/hosts call made by the datadog agent to retrieving the JSON in the _retrieve_json method in the file twistlock/datadog_checks/twistlock/twistlock.py and the code doesn't deal with pagination, so is truncated to only the first 50 results.

Jan 14 '22 22:01 kteague-tasktop

Hi @kteague-tasktop, thanks for reporting this issue. I'm currently working on a fix for this.

Feb 16 '22 16:02 fanny-jiang