Migrate File Backup Feature To Use IDX Instead Of 3Box

[think-in-universe]

Design

With the releases of IDX and Ceramic, we recommend replace 3Box-based file backup feature with IDX-based implementation.

In this solution, we followed the below design:

• identity and authentication: IDX
• storage: Ceramic

Related Issues

Fixes: https://github.com/ceramicnetwork/ceramic/issues/39

IDX Configuration

To save ETH Build files as private data, we used JWE for encryption/description of the files, following the description of the tutorial: https://blog.ceramic.network/how-to-store-encrypted-secrets-using-idx/

1. Create Schema and Definition
2. Include the IDX configurations into project

create schema

We used the JWE schema as described in the tutorial: https://blog.ceramic.network/how-to-store-encrypted-secrets-using-idx/

create definition

idx definition:create local --schema=ceramic://k3y52l7qbv1frygbtwkmqeh94snld52twral0ilijazxy1s38mlatem0wnmuj69ds --name="eth_build" --description="encrypted ETH.Build files"

idx-config.json

{
  "definitions": {
    "eth_build": "kjzl6cwe1jw1476v0ke4sb3n37jn18xwvcfx1vk6dewy1baqb8em5dnxoiknuj1"
  },
  "schemas": {
    "EncryptedFile": "ceramic://k3y52l7qbv1frygbtwkmqeh94snld52twral0ilijazxy1s38mlatem0wnmuj69ds"
  }
}

Acceptance Criteria

• [x] Should replicate both the "save to 3Box" and "load from 3Box" functionality currently found on eth.build
• [x] Should be a fully functional demo and ready to merge into Eth.Build pending production-readiness of IDX
• [x] Must be open source & contributed to Eth.Build
• [x] Should ensure that saved data will be persisted (e..g, on a Textile Bucket or SkyDB network)

Challenges

1. Here we used Ceramic as the storage solution to save files. We encountered an issue when using Ceramic because there's a 256KB size limit for a single document in Ceramic, and in ETH.BUILD, the size of saved screenshot of current status (via canvas.toDataUrl() function ) will as large as 500KB. To resolve this issue, we modified the image type (to jpeg) and quality of the screenshot to control the screenshot size within 64KB.
2. The IDX loading / writing speed is much slower than 3Box at the moment in my testing. May need more investigation about how to improve its performance.

Demo

Video: https://vimeo.com/493497686

Suppose that you have created something with Eth.Build, and then you'd like to save your file via IDX.

1. Click Save button, and select Save to IDX

2. Choose Wallet and connect to IDX via MetaMask

3. Input file name, and save into IDX + Ceramic

4. The files will be saved into IDX + Ceramic successfully

Next, if you leave Eth.Build for a while, and want to load the saved files via IDX when you visited the site again.

1. Click Load button, and select Load From IDX

2. Similar to Save workflow, you'll need to connect your account with IDX (if it hasn't been connected automatically yet)

3. In a few seconds, the saved files will appear in the dialog, and you can click anyone to resume the previous work.

Guideline for Migration from 3Box to IDX

Here is a quick reference for the DApp that wants to migrate from 3Box to IDX.

1. Learn the concepts: since IDX is different from 3Box in many ways. you may start with reading the doc: https://idx.xyz/docs
2. Authentication: replace openBox / openSpace / logoutBox with connectIDX / logoutIDX methods
3. Data Storage: review the data structure of your application. replace save data to public/private space with other solutions including Ceramic, OrbitDB, Textile ThreadsDB or SkyDB.
4. User Interface: replace 3Box icons, buttons, workflow, etc. with IDX resources

[think-in-universe]

@austintgriffith Hey Austin, could you please help review the PR and let me know for any feedback or suggestions?

[Aragar199]

@think-in-universe I resolved the merge conflicts to include these changes and am seeing an error.

Went to https://github.com/open-tribe/eth.build to build from master to see if it still works and am seeing the same error when trying to save or load from idx, can you help?

[grahamtallen]

#85