grafana
feature request: improve page-limits handling
we have some hardcoded page limits in the code, for example:
- https://github.com/grafana/github-datasource/blob/4452e594617f889d8cc846b03b02b1f17ffc5b2f/pkg/github/constants.go#L4
- https://github.com/grafana/github-datasource/blob/4452e594617f889d8cc846b03b02b1f17ffc5b2f/pkg/github/projects/projects.go#L15
we should increase these, but if we increase them too much we'll hit rate limits. some ideas:
- make these user-configurable
- warn the user if they are trying to fetch too much data
(see https://github.com/grafana/github-datasource/pull/232 for more context)
I ran into this yesterday. The project I was querying has hundreds of closed issues, so no matter what I did I only got the first bunch. It also seems the filters are applied after the query, so I couldn't work around the limit either 😞
Possibly related given apps have much higher rate limits compared to a user PAT. https://github.com/grafana/github-datasource/issues/88
Another thing that could help is using the logged in users auth since that could scale better for concurrent users compared to using app auth
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28
