mempool
Feature: track-mempool websocket subscriptions
Some common use-cases involve fetching a copy of every transaction that enters the mempool, e.g:
- syncing a local copy of the mempool with a Mempool backend.
- monitoring for new transactions matching certain criteria.
- running statistics on newly broadcast transactions.
Currently, the easiest way to achieve this via the Mempool API is to poll the /api/mempool/txids endpoint, calculate diffs between the results, and then fetch new txids individually from the /api/tx/<txid> endpoint.
But the /api/mempool/txids endpoint is extremely slow and expensive, especially when there are a very large number of transactions in the mempool.
This PR implements a much more efficient alternative via some new websocket subscriptions track-mempool and track-mempool-txids.
On every mempool update and new block, any websocket clients subscribed to track-mempool receive an object in the mempool-transactions field of the websocket response like:
{
...
"mempool-transactions": {
added: MempoolTransactionExtended[], // new transactions
removed: string[], // txids of transactions evicted from the mempool for non-block-inclusion reasons
mined: string[], // txids of transactions mined into the latest block
replaced: { replaced: string, by: MempoolTransactionExtended }[], // RBF replacements
}
}
Alternatively, websocket clients subscribed to track-mempool-txids receive an object in the mempool-txids field of the websocket response like:
{
...
"mempool-txids": {
added: string[], // txids of new transactions
removed: string[], // txids of transactions evicted from the mempool for non-block-inclusion reasons
mined: string[], // txids of transactions mined into the latest block
replaced: { replaced: string, by: string }[], // RBF replacements
}
}
The track-mempool subscription is potentially a lot of data, since it sends the full transaction objects for every new mempool transaction, although we do serialize the data once and reuse it across all subscribed clients which is fairly efficient.
If that's too much, we could lock the full track-mempool feature behind a config setting and only support track-mempool-txids by default. Then users would still need to make individual /api/tx/<txid> requests to get transaction data and be subject to the REST API rate limits etc. I think this would still be a huge improvement over polling api/mempool/txids.
