facebook
GitHub Rate Limit Crashes Sapling
I ran sl pr on a stack of 12 commits
Not too long later I ran sl ghstack and got the following error
Traceback (most recent call last):
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/__init__.py", line 106, in run
dispatch.run(args, fin, fout, ferr, config)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/dispatch.py", line 161, in run
status = (dispatch(req) or 0) & 255
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/dispatch.py", line 503, in dispatch
ret = _runcatch(req)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/dispatch.py", line 712, in _runcatch
return _callcatch(ui, req, _runcatchfunc)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/dispatch.py", line 721, in _callcatch
return scmutil.callcatch(ui, req, func)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/scmutil.py", line 147, in callcatch
return func()
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/dispatch.py", line 683, in _runcatchfunc
return _dispatch(req)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/dispatch.py", line 1218, in _dispatch
ret = runcommand(
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/ext/undo.py", line 146, in _runcommandwrapper
result = orig(lui, repo, cmd, fullargs, *args)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/ext/journal.py", line 80, in runcommand
return orig(lui, repo, cmd, fullargs, *args)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/ext/sparse.py", line 504, in _tracktelemetry
res = runcommand(lui, repo, *args)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/ext/copytrace.py", line 173, in _runcommand
return orig(lui, repo, cmd, fullargs, ui, *args, **kwargs)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/dispatch.py", line 913, in runcommand
ret = _runcommand(ui, options, cmd, d)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/dispatch.py", line 1259, in _runcommand
return cmdfunc()
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/dispatch.py", line 1217, in <lambda>
d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/util.py", line 1285, in check
return func(*args, **kwargs)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/ext/ghstack/__init__.py", line 51, in ghstack_command
return submit_cmd(ui, repo, *args, **opts)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/edenscm/ext/ghstack/__init__.py", line 104, in submit_cmd
ghstack.submit.main(
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ghstack/submit.py", line 199, in main
submitter.prepare_updates()
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ghstack/submit.py", line 968, in prepare_updates
self.process_new_commit(s, stack_index)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ghstack/submit.py", line 609, in process_new_commit
r = self.github.post(
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ghstack/github.py", line 40, in post
return self.rest('post', path, **kwargs)
File "/opt/homebrew/Cellar/[email protected]/3.8.15/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ghstack/github_cli_endpoint.py", line 38, in rest
raise RuntimeError(result.error)
RuntimeError: {
"message": "You have exceeded a secondary rate limit and have been temporarily blocked from content creation. Please retry your request again later.",
"documentation_url": "https://docs.github.com/rest/overview/resources-in-the-rest-api#secondary-rate-limits"
}
Wow, I have only encountered that when doing a lot of testing of Sapling/GitHub integration. Out of curiosity, did you have sl web / Interactive Smartlog open? It does poll for some amount of data to see if the status of the PRs in your stack have changed. Could that (or anything else) have been eating away at your quota?
I didn't
I did have some ReviewStacks open but pretty sure that would have been on a separate limit?
@spence-novata Hmm, my interpretation of this documentation is that it is 5,000 requests per hour across all of your OAuth tokens:
https://docs.github.com/en/developers/apps/building-github-apps/rate-limits-for-github-apps#default-user-to-server-rate-limits-for-githubcom
This makes sense in that you shouldn't be able to mint more OAuth tokens to give yourself more quota, I suppose? Do you know if you're using anything else besides Sapling and ReviewStack that would be using the token? Were you opening a lot of browser tabs with ReviewStack / were there a lot of files in the commits that you're working with?
ReviewStack does store quite a bit of data in IndexedDB locally to reduce the number of requests it makes, though I know there's more I could be doing there.
A few maximum, certainly not tens
I have some other apps integrating with GitHub,
GitLive: while it seems to do a lot of caching. I vaguely recall it doing a lot of work when it was first installed and wonder if for some reason, it just so happened to do a clean fetch, maybe a bunch of caches expired together.
SonarCloud is the other possible offender. Since I opened two stacks together, I wonder if the first one triggered a dozen CI builds, which triggered SonarCloud to do a bunch of requests from GitHub. Then when it came to the second stack the limits had already been reached.
In either case, it seems hard to imagine them managing to generate 5ks worth, but then I have no idea how poorly optimized any of them might be internally
I just tried creating PRs for 34 issues, and I also got a rate limit error:
(venv) flameeyes@Gamestation:~/repos/unpaper$ sl pr submit
pushing 37 to https://github.com/Flameeyes/unpaper
created new pull request: https://github.com/unpaper/unpaper/pull/124
created new pull request: https://github.com/unpaper/unpaper/pull/125
created new pull request: https://github.com/unpaper/unpaper/pull/126
created new pull request: https://github.com/unpaper/unpaper/pull/127
created new pull request: https://github.com/unpaper/unpaper/pull/128
created new pull request: https://github.com/unpaper/unpaper/pull/129
created new pull request: https://github.com/unpaper/unpaper/pull/130
created new pull request: https://github.com/unpaper/unpaper/pull/131
created new pull request: https://github.com/unpaper/unpaper/pull/132
created new pull request: https://github.com/unpaper/unpaper/pull/133
created new pull request: https://github.com/unpaper/unpaper/pull/134
created new pull request: https://github.com/unpaper/unpaper/pull/135
created new pull request: https://github.com/unpaper/unpaper/pull/136
created new pull request: https://github.com/unpaper/unpaper/pull/137
created new pull request: https://github.com/unpaper/unpaper/pull/138
created new pull request: https://github.com/unpaper/unpaper/pull/139
created new pull request: https://github.com/unpaper/unpaper/pull/140
created new pull request: https://github.com/unpaper/unpaper/pull/141
created new pull request: https://github.com/unpaper/unpaper/pull/142
created new pull request: https://github.com/unpaper/unpaper/pull/143
abort: error creating pull request for 7a78d78e9c635718a9a50c138aafb49001cc9bee: {
"message": "You have exceeded a secondary rate limit and have been temporarily blocked from content creation. Please retry your request again later. If you reach out to GitHub Support for help, please include the request ID E22E:334AB6:15EF231:1638E46:65A42E07.",
"documentation_url": "https://docs.github.com/rest/overview/rate-limits-for-the-rest-api#about-secondary-rate-limits"
}
So I'm afraid sapling should be more conservative with this :(
