cloudbeat
cloudbeat copied to clipboard
Account for rate limiting in GCP fetchers
Motivation
GCP fetchers mainly use the assets inventory api via the ListAssets method, which has the following quotas:
-
100per minute per consumer project -
800per minute per organization -
650,000per day per organization
for both our use cases - single-account and organization-account, we always use the same project to consume the quota, as we set it once, when deploying an agent. (can be verified with gcloud config get billing/quota_project). this means we can use just the per-project quota for rate limiting.
Definition of done
- GCP assets inventory's
ListAssetsmethod usage does not exceed the default quota (100 per minute per project)
Out of scope
- synced cloudbeat instances consuming the same quota