ouroboros-network icon indicating copy to clipboard operation
ouroboros-network copied to clipboard

Immutable DB: provide Reader-like functionality

Open mrBliss opened this issue 6 years ago • 1 comments

Iterators can be used to stream from the ImmutableDB. However, upon initialisation, the bounds of an iterator are fixed, even when the upper bound is open. This means that blocks added after the iterator was opened will not be included in its stream.

The ChainDB has iterators with the same behaviour, but also Readers, which do include newly added blocks. These are implemented on top of ImmutableDB iterators. To simplify their implementation and to avoid having to reopen iterators (when reading from the tip of the ImmutableDB while new blocks are being added, we have to reopen the iterator to include these blocks), which is not cheap (a block is read every time), a part of this complexity could be shifted to the ImmutableDB.

A possible API:

data Reader hash m a = Reader {
      readerInstruction         :: m (Maybe a)
    , readerInstructionBlocking :: m a
    , readerClose               :: m ()
    , readerId                  :: ReaderId
    }
  deriving (Functor)

where a will be instantiated with ByteString.

mrBliss avatar Jul 15 '19 08:07 mrBliss

We're going to provide a 30 to this ticket. This is based on the fact that at the moment the consensus team doesn't fully grasp it. We can create a separate tech-debt item to follow up on the understanding / complexity of this overall task.

jackrousian avatar Jul 19 '22 15:07 jackrousian