flow-go icon indicating copy to clipboard operation
flow-go copied to clipboard

Khalil/1716 unicast rate limiter

Open kc1116 opened this issue 3 years ago • 1 comments

This PR defines 2 rate limiters for messages sent via unicast. Both rate limiters are configurable on p2p middleware, and are used in the unicast stream handler.

  • bandwidth rate limiter - rate limits unicast message bandwidth allowed per some configured interval
  • streams rate limiter - rate limits amount of streams that can be created per some configured interval

Each rate limiter keeps track of current rate limited peers and the rate limiter interface provides a func IsRateLimited to check if a peer is currently being rate limited.

This PR also improves the connection gater by adding the ability to configure multiple peer filters for outgoing and incoming intercepted connections. This allows us to add the IsRateLimited func as a peer filter on the connection gater. The peer manager peer provider func topologyPeers on the middleware was also updated to filter topology peers through configured peerManagerFilters before returning the list of peers to the peer manager. Both of these improvements allow us to provide connection gating and disconnects for rate limited peers in the correct lifecycle of a connection respective to the networking layer.

kc1116 avatar Jul 21 '22 17:07 kc1116

Codecov Report

Merging #2840 (71ef077) into master (860f647) will increase coverage by 1.06%. The diff coverage is 39.51%.

@@            Coverage Diff             @@
##           master    #2840      +/-   ##
==========================================
+ Coverage   54.31%   55.37%   +1.06%     
==========================================
  Files         610      761     +151     
  Lines       53161    70024   +16863     
==========================================
+ Hits        28872    38773    +9901     
- Misses      21922    28054    +6132     
- Partials     2367     3197     +830     
Flag Coverage Δ
unittests 55.37% <39.51%> (+1.06%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
cmd/scaffold.go 16.89% <0.00%> (-1.11%) :arrow_down:
network/p2p/connection/connector.go 68.42% <ø> (ø)
network/p2p/p2pnode/libp2pNode.go 28.01% <ø> (ø)
network/p2p/unicast/ratelimit/noop_rate_limiter.go 0.00% <0.00%> (ø)
network/p2p/unicast/ratelimit/rate_limiters.go 0.00% <0.00%> (ø)
utils/unittest/timer.go 0.00% <0.00%> (ø)
utils/unittest/unittest.go 7.50% <ø> (ø)
...rk/p2p/unicast/ratelimit/bandwidth_rate_limiter.go 77.50% <77.50%> (ø)
...work/p2p/unicast/ratelimit/message_rate_limiter.go 77.50% <77.50%> (ø)
...ratelimit/internal/limiter_map/rate_limiter_map.go 78.66% <78.66%> (ø)
... and 158 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov-commenter avatar Jul 28 '22 00:07 codecov-commenter

bors merge

kc1116 avatar Oct 25 '22 17:10 kc1116

Build failed (retrying...):

bors[bot] avatar Oct 25 '22 17:10 bors[bot]

Canceled.

bors[bot] avatar Oct 25 '22 17:10 bors[bot]

bors try

kc1116 avatar Oct 25 '22 17:10 kc1116

try

Build failed:

bors[bot] avatar Oct 25 '22 18:10 bors[bot]