cloudbeat icon indicating copy to clipboard operation
cloudbeat copied to clipboard

Account for rate limiting in Azure fetchers

Open orouz opened this issue 1 year ago • 0 comments

Motivation

we need to account for rate limiting in our Azure fetchers to avoid losing resources we want to evaluate. our usage of azure go sdk can be divided to two:

  1. graph explorer
    • there is no easy way to figure out rate limiting as different queries may end up consuming different quotas. instead, we should verify our usage adheres to azure's throttling guidelines.
  2. all other clients
    • implement rate limiting when applicable

Definition of done

  • account for rate limiting when possible
  • add a retry (with backoff) mechanism for failed requests

Out of scope

  • synced cloudbeat instances consuming the same quota

orouz avatar Apr 01 '24 08:04 orouz