oddsdk
Implement File System Access for high performance user experience.
Summary
Implement File System Access for high performance user experience.
Problem
ipfs is seriously slow, especially when dealing with directories with many files.
Impact
The impact of not having this feature is that wnfs will continue to be crippled by ipfs slowness
Solution
Use the fastest storage possible in the simplest way, and let the web workers slowly and gracefully deal with syncing to ipfs.
An interesting idea.
So far we've been using IndexedDB as the storage backend though js-ipfs. We're slowly working on depending on more granular parts of js-ipfs instead of the whole stack at once which will help us test out replacements for some of these parts in js-ipfs, as well as some useful in-memory caching strategies in rs-wnfs, which already show improved performance.
Thanks for the links! As IndexedDB is notorious for being slow, it may be interesting to see how well a blockstore based on high-performance file system access may work.
Hey @gotjoshua — yeah I would like to see us support this.
Needs some discussion around use cases and of course restricted to ONLY Chrome browsers at the moment. For example — we could do a high performance desktop sync client rather than rely on the browser, but that’s sort of a different model.
we are in fact heavily involved with the IPFS protocol itself. Our plan is to move away from js-IPFS in browser (that is mostly the slow part).
CAR files and HTTP transport, CAR Mirror, and various other things are what we’re looking at.
Let’s keep this open as the File System Access issue and workshop this a bit more. I’ll start a forum thread for this.
I'm looking into these two libs so far:
https://github.com/jimmywarting/native-file-system-adapter/
https://github.com/GoogleChromeLabs/browser-fs-access
anyone know of others worth considering?
I did some tests today.
It seems that the file system access api is very impractical to use in worker contexts.
Possible but rather impractical (especially for writing new files).
The older navigator.storage api has >90% availability : https://web.dev/file-system-access/#accessing-files-optimized-for-performance-from-the-origin-private-file-system
Could be a better choice (especially if rs-wnfs can access it natively or conveniently via a webworker)