cloudbeat
cloudbeat copied to clipboard
Account for rate limiting in Azure fetchers
Motivation
we need to account for rate limiting in our Azure fetchers to avoid losing resources we want to evaluate. our usage of azure go sdk can be divided to two:
- graph explorer
- there is no easy way to figure out rate limiting as different queries may end up consuming different quotas. instead, we should verify our usage adheres to azure's throttling guidelines.
- all other clients
- implement rate limiting when applicable
Definition of done
- account for rate limiting when possible
- add a retry (with backoff) mechanism for failed requests
Out of scope
- synced cloudbeat instances consuming the same quota