blockparser icon indicating copy to clipboard operation
blockparser copied to clipboard

Realtime parser ideas

Open luisbebop opened this issue 10 years ago • 3 comments

Nice work. Your code is super clean, simple and fast.

I'd like to ask you some ideas about how to implement real time parser of new transactions in parser.cpp For example a callback to read all tx with a determined OP_RETURN pattern and then keep looping showing new transactions with that pattern.

What do you think is the best way to implement it? I don't need specific details just some hints from your side about what would be the optimal way.

luisbebop avatar Oct 18 '15 17:10 luisbebop

i'd be interested in realtime parsing of new blocks and persistent sqlite database. Maybe i'll try work on it and submit a PR

6coind avatar Nov 05 '15 14:11 6coind

The problem with doing this is longest chain validation. At start time, blockparser computes the longest chain, which is easy because the code assumes a static blockchain. For a "real time" parser, it would have to deal with new blocks on the fly, and some of these may eventually become invalid (if they're not part of the longest chain). When this happens, you'd have to revert all internal data structures back to the time of the fork ... you'd also have to revert all transactions commited to the sqlite db. This is not easy at all.

znort987 avatar Nov 13 '15 13:11 znort987

What you want to do is called complex event processing (CEP). Take a look at Esper. You can do that with Esper and a bit of Java.

BTW, one thing I'd like to see in this blockchain parser is a bloom filter option for check if a public address has been seen.

qwerty55 avatar Mar 16 '16 07:03 qwerty55