node-id3 icon indicating copy to clipboard operation
node-id3 copied to clipboard

Support huge files

Open kentcdodds opened this issue 4 years ago • 2 comments

I bumped into this:

(node:2830) UnhandledPromiseRejectionWarning: RangeError [ERR_FS_FILE_TOO_LARGE]: File size (4280247303) is greater than 2 GB
    at tryCreateBuffer (fs.js:352:13)
    at Object.readFileSync (fs.js:388:14)
    at Object.module.exports.write (./book-stitcher/node_modules/node-id3/index.js:48:27)
    at createBook (./book-stitcher/bin.js:124:26)

My file is pretty large. I tried looking through the code of node-id3 and I don't think writing the ID3 tags using streams (rather than reading the whole file at once) is possible with the current features. I also don't know if that would be possible generally.

Any tips and ideas would be welcome.

kentcdodds avatar Feb 10 '21 20:02 kentcdodds

Hey, you're right, there need to be some changes in order for write to work, I've created a new branch where reading by file stream is implemented: https://github.com/Zazama/node-id3/commit/a486c2a730a421b735f85563c3766d3952a405f6

Still todo:

  • [ ] Implement padding
  • [ ] Write by stream
    • [ ] If tag smaller than new data: exchange chunks by new ones in order to avoid storing in memory
    • [ ] If tag bigger than new data: Overwrite and use padding instead of writing full file

Zazama avatar Feb 11 '21 02:02 Zazama

That's awesome! Thanks for being willing to work on this 💖

kentcdodds avatar Feb 12 '21 14:02 kentcdodds

Let's keep #161

Zazama avatar Apr 05 '23 11:04 Zazama

Lol, I forgot that I'd already opened an issue about this 🤦‍♂️ sorry for the extra noise

kentcdodds avatar Apr 05 '23 12:04 kentcdodds