ZstdSharp icon indicating copy to clipboard operation
ZstdSharp copied to clipboard

GetDecompressedSize reports wrong size from stream compressed array.

Open mateli opened this issue 8 months ago • 3 comments

When using CompressorStream to create a compressed byte array running GetCompressedSize returns the wrong number. However compressing the same source data using Compressor.Wrap produces the correct size. Is the storage format different?

mateli avatar May 13 '25 19:05 mateli

GetDecompressedSize returns the upper bound for the decompressed size, see ZSTD_decompressBound. You can use SetPledgedSrcSize to specify the source size in streaming mode, in which case GetDecompressedSize will return the exact value.

Is the storage format different?

https://facebook.github.io/zstd/zstd_manual.html#Chapter15:

Decompressed size is an optional field, that may not be present, especially in streaming mode

oleg-st avatar May 14 '25 08:05 oleg-st

So I manually keep track of written bytes and call this before closing the stream/file? Is it possible to update files that has already been written with a correct size?

mateli avatar May 17 '25 16:05 mateli

SetPledgedSrcSize can be called before the frame header is written.

Is it possible to update files that has already been written with a correct size?

I don't think there's an API for something like this. You can forward your question to the authors of zstd.

oleg-st avatar May 19 '25 10:05 oleg-st