FiniteStateEntropy icon indicating copy to clipboard operation
FiniteStateEntropy copied to clipboard

FSE_compressU16() computes clipped data if dstCapacity different

Open JohanKnutzen opened this issue 7 years ago • 1 comments

Repro:

constexpr size_t inSize = 24;
uint16_t in[inSize] =
  {0, 0, 3, 2, 0, 0, 0, 0, 314, 0, 0, 0, 0, 51, 50, 0, 0, 59, 22, 36, 0, 55, 32, 22};
unsigned int maxSymbolValue = 314;
unsigned int outSize = 48;

uint8_t out1[256];
uint8_t out2[256];
size_t        numBytes1 = FSE_compressU16(out1,
                                   outSize,
                                   in,
                                   inSize,
                                   maxSymbolValue,
                                   0);

size_t numBytes2 = FSE_compressU16(out2,
                                   256,
                                   in,
                                   inSize,
                                   maxSymbolValue,
                                   0);
// numBytes1 is now 34 bytes
// numBytes2 is now 43 bytes

FSE_decompressU16(in, inSize, out2, numBytes2);
FSE_decompressU16(in, inSize, out1, numBytes1); // <- Crashes 

When output bytes is 48 rather than 256 as an argument for FSE_compressU16(), all generated bytes are identical up until 34 bytes. The rest of the bytes are clipped. The difference in code paths is that in the 34 bytes case, the non-fast path is chosen.

JohanKnutzen avatar Jan 03 '19 01:01 JohanKnutzen

Is this fixed in dev? I can test it if it is believed to be fixed.

JohanKnutzen avatar Aug 19 '20 20:08 JohanKnutzen