IGNITE-17029 PITR: Consistent Cut

[timoninmaxim]

Consistent Cut is an algorithm for taking consistent distributed snapshot. The Cut is a point on timeline such that set of committed transactions before (after) this point is the same on every node in a cluster. Order of transactions matters: if one transaction (A) depends on previous one (B) - then (A) can't be before the Cut when (B) is after: [A, Cut, B]. Right orders of transactions: [B, A, Cut] or [B, Cut, A].

Transaction can be described as ordered sequence of events: messages between nodes (PrepareRequest, PrepareResponse, FinishRequest) and local TransactionStates. Correct order of events is well known, it depends on transaction algorithm (TwoPhaseCommit, OnePhaseCommit).

Goal is to track whether every event within single transaction on the same side (before or after) of the Cut. To achieve this:

1. The messages between nodes are signed with latest known local Cut version.
2. On receiving message with new Cut version node has to handle new version before applying the message (start Consistent Cut procedure locally).
3. Consistent Cut procedure consist of steps: update local Cut version, find which transactions aren't awared of versions of previous events, write Cut info to WAL.
4. While receiving messages such transactions become aware of the versions. After all such transactions completed with known Cut version a node writes second Cut info to WAL.

This algorithm guarantees that every node has the same committed transactions before Cut in the same order.

More info in IEP-89 PITR