diehard icon indicating copy to clipboard operation
diehard copied to clipboard

allow :rate-limiter/rate to be floats, to have floating points rates (0.5 => 1 req every 2 seconds)

Open realgenekim opened this issue 2 years ago • 3 comments

Thank you for such a wonderful library — I am using this library so often when I hit external API endpoints.

Recently, I started using it to query the OpenAI API, which has token limits that often push the rate limit below one request per second. (I use thread pools via claypoole library to fire off parallel requests)

I found that it was super-easy to specify rate limits of less then 1 req/sec, merely by specifying floats less then 1.0. All I had to do was relax the spec to take number? instead of int?.

PS: I haven't done any formalized or methodical testing, beyond just observation, but it works well enough for my purposes. (Before opening up this PR, I did some more measurements. A rate limit of 0.5 will fire 10 calls in 25 seconds. 0.33 will take 36 seconds. Good enough for me!)

realgenekim avatar Jul 05 '23 16:07 realgenekim

Oops. Forgot to modify documentation in README.md. (I hope I'm doing this correctly — I haven't done this very often!)

realgenekim avatar Jul 05 '23 16:07 realgenekim

Thank you @realgenekim ! I didn't realize we can have such little rate :). Could you please add a test case in tests/diehar/rate_limiter_test.clj by replacing the current test with a float point rate so that we can verify this behaviour

sunng87 avatar Jul 06 '23 14:07 sunng87

@realgenekim Are you going to finish this PR? This is a worthwhile addition to the lib

kevinmershon avatar Dec 04 '23 20:12 kevinmershon

Hi, @kevinmershon and @sunng87 — sorry for the delay on this. I updated the PR that includes a test. (My apologies in advance if I did something wrong. Just let me know, and I will fix.)

realgenekim avatar Feb 15 '24 20:02 realgenekim

LGTM. Thank you for following up.

sunng87 avatar Feb 16 '24 10:02 sunng87