node-id3 icon indicating copy to clipboard operation
node-id3 copied to clipboard

Move file reading and writing to streaming

Open Zazama opened this issue 2 years ago • 0 comments

This PR implements file streaming for reading files using the NodeID3.read, NodeID3.write (and update) function. By doing so, we support reading/writing big files and go easy on memory.

  • We read/write the file in chunks of max. 20 megabytes (if you run node.js, you probably have 20mb to spare)
  • The reader will automatically detect the start of an ID3 Tag and extract it fully into a buffer
    • This assumes that ID3 Tags are small enough to fit into memory
    • If it were bigger, the reading would fail anyways, because the parsed information would not fit into memory
  • Code is slightly repeated because of sync/async support. I haven't found a better way yet
  • On write, we create a temp file next to the original one
    • This is done using the "tmp" module because we need collision handling and I think it's good to keep the files in the same folder so it can be found easily on failure.
    • First, the new ID3 Tag is written at the beginning
    • Then the rest of the old file is streamed into the new one. If an old ID3 Tag is found, it is skipped
  • The type definition of a NodeID3.write callback should not return a buffer if a file is being manipulated. This is also how it is documented in the README.md

Related issue: https://github.com/Zazama/node-id3/issues/161

Zazama avatar Apr 06 '23 21:04 Zazama