redpanda icon indicating copy to clipboard operation
redpanda copied to clipboard

Separate Token bucket algorithm from recovery throttle

Open ZeDRoman opened this issue 3 years ago • 2 comments

Cover letter

Token bucket algorithm will be used later in controller log rate limiting. For it we need to refactor code and add method for additional usage

Refactor recovery throttle. Separate Token bucket algorithm to separate class Add try_throttle method to Token bucket algorithm Add support of burst throttling to Token bucket algorithm

ZeDRoman avatar Sep 19 '22 11:09 ZeDRoman

do you think that utils/ would be a more appropriate location than model/?

I thought that Token bucket performs some separate logic. Util is something that performs a small specific task. Maybe I have the wrong understanding of separation there. Can you explain please

ZeDRoman avatar Sep 20 '22 08:09 ZeDRoman

I thought that Token bucket performs some separate logic. Util is something that performs a small specific task. Maybe I have the wrong understanding of separation there. Can you explain please

The model directory is primary for redpanda-specific type definitions and some heavier weight things like record batch. It's much more closely related to the kafka-ness of redpanda.

Token bucket algorithm is a generic tool that can be used in many places and really isn't redpanda-specific.

dotnwat avatar Sep 21 '22 01:09 dotnwat

also, regarding multiplier. conclusion from conversation this morning with michal and rob and roman was that rate*multiplier was effectively the capacity of the bucket. action item is to change parameters to bucket to be (rate, capacity).

dotnwat avatar Sep 27 '22 22:09 dotnwat