hyperspace icon indicating copy to clipboard operation
hyperspace copied to clipboard

Allow to track sparsely synced hypercores

Open Frando opened this issue 5 years ago • 0 comments

Currently, the daemon emits an append event if new blocks are appended to a hypercore (either locally or by a remote for synced cores). However, no event is emitted when new blocks are downloaded for a sparsely synced hypercore.

I'd like to explore kappa-style apps on top of hyperspace that would index all downloaded blocks on a set of hypercores. For that to work efficiently, I think two things would be needed:

  • [ ] OnDownload method for the hypercore service that is emitted for each block downloaded with { id, seq }. Maybe this method should only be called if requested for a hypercore? So WatchDownloads({ id }), UnwatchDownloads({ id }) and OnDownload({ id, seq }). Or just emit it always, as OnAppend is at the moment. See #5
  • [ ] A way to quickly access which blocks have been downloaded for a hypercore, so maybe a GetBitfield or similar. This can be worked around by iterating and calling Has for each seq, which might be good enough. (This will be needed on each startup of the indexer client, because it cannot know if download events happened while it wasn't running). See #7

Frando avatar Jul 11 '20 10:07 Frando