eudico
eudico copied to clipboard
filecoin-project
Pull based mempool
In most Mine functions we use polling in the default clause of the select statement. This leads to the following million lines in the logs and probably big inefficiency.
2022-08-27T16:09:33.424Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000081971}
2022-08-27T16:09:33.424Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000091091}
2022-08-27T16:09:33.424Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000077593}
2022-08-27T16:09:33.424Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000106447}
2022-08-27T16:09:33.425Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.0001174}
2022-08-27T16:09:33.425Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000089388}
2022-08-27T16:09:33.426Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000085548}
2022-08-27T16:09:33.426Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000086867}
2022-08-27T16:09:33.427Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000084708}
2022-08-27T16:09:33.427Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000087421}
2022-08-27T16:09:33.427Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000087834}
2022-08-27T16:09:33.427Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000084959}
2022-08-27T16:09:33.428Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000081306}
2022-08-27T16:09:33.428Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000129739}
2022-08-27T16:09:33.429Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000084564}
2022-08-27T16:09:33.429Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000087551}
2022-08-27T16:09:33.429Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.00008116}
2022-08-27T16:09:33.429Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000089143}
2022-08-27T16:09:33.430Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.00007906}
2022-08-27T16:09:33.430Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.00007827}
2022-08-27T16:09:33.431Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000081545}
2022-08-27T16:09:33.431Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000085226}
2022-08-27T16:09:33.431Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000080777}
2022-08-27T16:09:33.431Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000088139}
2022-08-27T16:09:33.432Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.000077117}
2022-08-27T16:09:33.432Z INFO messagepool messagepool/selection.go:472 message selection done {"took": 0.0000788}
Could we improve it at least from a usability perspective:
- do not log these message selections with INFO level
- replace select-default to ticker-based approach
<-receiveFromMempool.C, but in this case, it is not obvious how often we should read from mempool - add notifications to mempool and use pulling in the Mine
The first suggestion is the most trivial one.
@adlrocha What do you think?
This is a great point, and I agree we should get it fixed.
The first option is the friction-less one (and the most obvious).
I think the the other two may have their trade-offs:
- A ticker-based approach may not be a bad idea, but there's a risk that we may not be able to choose the right ticker time. I think this should be consensus-specific and configurable.
- A push-based mempool is the most efficient alternative, but in this case we may still need some kind of ticker for consensus algorithms that support empty blocks (we may still need to propose new blocks even if there are no messages in the mempool).
I'd suggest to start with 1, document 2 and 3 in an issue, and decide after we re-architect the code. WDYT?
The first option added - https://github.com/filecoin-project/eudico/pull/234/commits/6fb898b5a42ab39b075c404d2c003b55e2e36777