disnake icon indicating copy to clipboard operation
disnake copied to clipboard

feat(http): improve HTTPClient rate limiting behavior

Open alentoghostflame opened this issue 2 years ago • 0 comments

Summary

A port of Nextcord's HTTP-Recore PR

DUE TO THIS BEING A PORT, I PROBABLY SCREWED SOMETHING UP! The code in this PR was modeled after NC's HTTPClient, but I can see that there are some obvious differences between DS's and NC's HTTPClient code. While I have tried to keep the changes intact or otherwise adapt them, I'm sure there are more subtle (and major) differences that I missed. If you think that I may have clobbered something, I probably didn't notice the change and did clobber it. Please say something!

Massively reworks the core of HTTPClient, adding support for:

  • Multiple authorization tokens/types, opening the way for OAuth2 (extensions?),
  • Global rate limit per auth to keep within the (default 50) maximum requests per second,
  • Proper rate limit bucket handling, allowing multiple requests in a single bucket to be ran simultaneously. Works nicely alongside asyncio.gather()!

Due to the major changes there are breaking changes, notably:

  • Documentation and comments will likely need to be updated across the board,
  • MaybeUnlock has been removed and replaced with a completely different implementation of RateLimit,
  • HTTPClient.request() will no longer clobber some given headers, such as auth and user-agent ones,
  • HTTPClient.token has been removed due to auth changes and Client._token has been added,
    • If necessary, HTTPClient._default_auth exists.
  • Many of HTTPClient's attributes are gone with new and old ones being _'d,
  • HTTPClient.static_login() no longer takes a token ("<token>") and instead takes an auth string. ("Bot <token>")

I'll finish fleshing out this PR summary well before I undraft it.

Checklist

  • [X] If code changes were made, then they have been tested
    • [ ] I have updated the documentation to reflect the changes
    • [X] I have formatted the code properly by running pdm lint
    • [X] I have type-checked the code by running pdm pyright
  • [ ] This PR fixes an issue
  • [ ] This PR adds something new (e.g. new method or parameters)
  • [X] This PR is a breaking change (e.g. methods or parameters removed/renamed)
  • [ ] This PR is not a code change (e.g. documentation, README, ...)

alentoghostflame avatar Jul 20 '23 01:07 alentoghostflame