ic icon indicating copy to clipboard operation
ic copied to clipboard

chore: [MR-620] Keep extra in-memory states

Open ShuoWangNSL opened this issue 1 year ago • 2 comments

The is the first PR towards removing in-memory state more eagerly at checkpointed heights. Currently, in-memory state at the previous checkpoint height is kept until the next CUP, roughly the whole checkpointing interval. We plan to remove it more eagerly and only keep it until the current CUP.

Currently,remove_inmemory_states_below purges states based on the certified height referenced in the finalized tip. As certified height could go beyond the summary block height, we need to keep certain states for several rounds more until the current CUP is created. Thus Consensus needs to inform state manager of the extra heights to keep through a new API .

This PR extends the interface, adds implementation and tests in state manager. Next, Consensus will use this API with proper arguments, i.e. filling in all the heights required for CUP creation and validation, and run end-to-end tests. After these two PRs, we can proceed with removing in-memory states more eagerly at checkpointing heights.

ShuoWangNSL avatar Oct 15 '24 14:10 ShuoWangNSL

Could you write in the description, why do we need "extra" states? It looks wasteful in terms of RAM otherwise

pakhomov-dfinity avatar Oct 24 '24 16:10 pakhomov-dfinity

Could you write in the description, why do we need "extra" states? It looks wasteful in terms of RAM otherwise

Sure. Updated the PR description.

ShuoWangNSL avatar Oct 24 '24 22:10 ShuoWangNSL