Real-Time Stream of Blocks API

[LLFourn]

Description

Many Bitcoin applications need to react in real-time to things happening on the blockchain. A core way of covering all those situations is being able to subscribe to a feed of all new blocks (and perhaps old ones too!). We do not currently have real-time APIs in BDK so your task is to create the first one (in a seperate crate for now).

Expected Outcomes

1. Design an API for subscribing to new Bitcoin blocks
   1. The API should allow you to stream all new blocks.
   2. The data should be returned as a Stream
   3. Decide whether the API should be a Stream of Block or BlockHeader (or both!).
   4. If BlockHeader then provide a way to get the full Block
   5. See if you can make the API take a block height so the stream returns all blocks after that one by going back and fetching old ones if needed.
2. Implement the API for some block source:
   1. bitcoin core rpc + zeroMQ
   2. bitcoin p2p (hard)
   3. mempool.space's websocket API (easy)
3. (optional) extend the API to allow users to monitor for:
   1. When a transaction is seen
   2. When a transaction is confirmed
   3. When an output is spent (and confirmed spent)
   4. transactions related to a certain address Note this can be done just by scanning through each block you get in your stream.
4. Provide the API in an external crate with a name like bdk_notify or whatever you choose!

Resources

You'll need to get understanding the basics of Bitcoin, Blocks, transactions and UTXOs etc. The next best resource would be to poke around https://mempool.space and https://mempool.space/api to see what data is available about the blockchain and in what form.

Skills Required

• Need to know the rust language to the extent that you can use the API of rust-bitcoin.
• Be comfortable with using (or learning how to use) reqwest
• Be comfortable with using (or learning how to use) async/await and futures in rust
• Be comfortable with using (or learning how to use) serde (for JSON deserialisation)
• BDK specific knowledge is not required.

Mentor(s)

• Lloyd Fournier (@LLFourn)

Difficulty

Medium

Competency Test (optional)

• Write a rust program to subscribe to the websocket API of mempool.space and print out any new blocks in real time.

[LLFourn]

So after thinking about it. I recall that @johncantrell97 mentioned that LDK made use of this API which also notified you of which blocks have been disconnected as well as connected.

So instead of a stream of block headers you'd have two types of events:

pub enum Event {
    Connected(BlockHeader),
    Disconnected((u32, BlockHash)) // height and hash
}

This would fit in well also with what I am doing with bdk_core. Does this make sense @johncantrell97?

[notmandatory]

Closing this issue since we don't have anyone to mentor it this year, can reopen next year if we have more folks who can mentor.