conserve
conserve copied to clipboard
store tiny files inline in the index
Files under about 100 bytes in length could be better stored directly in the index:
- the file content is potentially smaller than a hash referring to its block
- it avoids a separate IO for a data block when reading and writing, and probably still helps even after #66
There is a complication that the 0.6 index format is encoded in json, which can't directly include binary content. Options here seem to be:
- Only inline files whose content is a UTF-8 string. Many small files
- base-64 escape the content (not very nice)
- Wait to transition to a non-json index encoding, perhaps CBOR