v2

[TkTech]

• [ ] Refactor locks
  • [ ] Expose locking internals as first class features, allowing clearing locks and checking state from user code.
  • [ ] Expose a flag to disable early locking (at apply/delay time) and use only late locking (when the task actually starts) which enables recursive locking tasks.
• [ ] Strong typing for all settings and options, prevent typo's in the heimdall dict.
• [ ] Allow manual control of RateLimit objects, such as self.heimdall.rate_limit.offset(timedelta(minutes=15))

[ThierryDeruyttere]

Hi,

Nice package! This does some of the things that I was looking for. If I could make a suggestion, for my task it'd be pretty useful if I could dynamically update my rate limitter. I.e. I'm using fb api and I have a 200 req/user/hour limit. But people could be connecting their fb account with my application so my rate limit goes up actually. It'd be nice if we could just start off with a 200 req/hour limit but everytime the task is ran, check if we can actually increase our rate limit to for instance 400 req/hour.

[TkTech]

Hi @ThierryDeruyttere. You should be able to do this with the dynamic rate limiting, see https://github.com/TkTech/celery-heimdall#dynamic-rate-limiting. The RateLimit object can take a lambda, which can be whatever you want. You can pluck it out of a database or change it based off time of day for example.

This hasn't been tested as much, so if you run into an issue just make a ticket.

[ThierryDeruyttere]

Aha I read over that part, thanks!

[ThierryDeruyttere]

I noticed that the current rate limiter does kind of have a bursting side to it? Maybe give people the option to have a uniform distribution when making calls?