hyperdrive icon indicating copy to clipboard operation
hyperdrive copied to clipboard

Undocument `drive.clear`

Open mixmix opened this issue 4 years ago • 3 comments

Hi I've been working on how to remove files from my hyperdrive. Have looked across the some issues and this PR and seems like drive.clear is what I'm looking for

couple of questions:

  1. any reason this is undocumented? (other than the usual - time!)

  2. can I use hyperdrive.clear with storage: random-access-file?


I see random-access-file has a file.del which is likely called by hypercore.clear, but raf also says of file.del

"Will truncate the file if offset + length is larger than the current file length. Is otherwise a noop."

To me this suggest that unless I'm deleting the most recent files, nothing will happen? I'm running some tests and not sure I'm seeing my storage folder reducing in size after a clear (using linux du -d 1 storagePath)

(apologies this question spans repos, I'm still learning the stack)

mixmix avatar Jun 09 '21 10:06 mixmix

cc @okdistribute - I'm testing code magma collective wrote. I think they might have actually found about this API from you!

mixmix avatar Jun 09 '21 10:06 mixmix

excitingly I have file which has different sizes based on how you ask now:

➜ ls -lah a3/8f/a38f5d6a076fbfb7d1830886822aad4de0b7feb0de8b96b255d277a5a4f49995/data 
-rw-rw-r-- 1 mix mix 2.2G Jun 21 15:41 a3/8f/a38f5d6a076fbfb7d1830886822aad4de0b7feb0de8b96b255d277a5a4f49995/data

2.2GB

➜ du -h a3/8f/a38f5d6a076fbfb7d1830886822aad4de0b7feb0de8b96b255d277a5a4f49995/data
901M    a3/8f/a38f5d6a076fbfb7d1830886822aad4de0b7feb0de8b96b255d277a5a4f49995/data

901MB

Reading around I'm learning about "apparent disk size", as in file size !== disk usage. If I've read this right we have a very large sparse file (has lots of empty zeros) so while the "apparent size" is 2.2GB it's only actually taking up 901MB of space currently?

Having a hard time knowing what to read to report disk usage to users (and to assess how pruning files is going)

mixmix avatar Jun 21 '21 04:06 mixmix

I think it's undocumented because nobody got around to documenting it. 😅

I've been using it in production and it seems to work decently with random-access-file.

Sparse mode is more stable on Windows as of this release: https://github.com/random-access-storage/random-access-file/pull/24

RangerMauve avatar Jun 23 '21 21:06 RangerMauve

Documented here : https://github.com/holepunchto/hyperdrive#const-cleared--await-driveclearpath-options

mixmix avatar May 28 '23 22:05 mixmix