lemire
wrong size for the recovered integer array
I believe There is mistake in setting the size of the recovered int[] to ChunkSize in the advancedExample() method in https://github.com/lemire/JavaFastPFOR/blob/master/example.java,
int TotalSize = 2342351; // some arbitrary number
int ChunkSize = 16384; // size of each chunk, choose a multiple of 128
int[] data = new int[TotalSize];
int[] recovered = new int[ChunkSize];
If I do the following at the end:
if(Arrays.equals(data, recovered)) {
System.out.println("Elevation data is recovered in memory without loss");
}
else
throw new RuntimeException("bug"); // could use assert
It would throw me the RuntimeException, since the data array and the recovered array are different and NOT of the same size.
https://github.com/lemire/JavaFastPFOR/blob/1d650e40f6ce3052042871161488971052c9fd32/example.java#L240-L243
Please provide a complete and reproducible code sample.
If you think that the example exactly as written is wrong, say so. If it is some other example that you have written that is not working as you expect, please provide the full example.
Please provide all necessary details for reproduction.
The provided snippet looks weird, as we compare the initial data (size=2342351) with a chunk of the restored data (size= 16384).
The provided snippet looks weird, as we compare the initial data (size=2342351) with a chunk of the restored data (size= 16384).
Please elaborate.
Here is the current output...
Compressing 2342351 integers in one go
compressed from 9149KB to 357KB
data is recovered without loss
Compressing 2342351 integers in one go using the headless approach
compressed from 9149KB to 357KB
data is recovered without loss
Compressing 2342351 integers using chunks of 16384 integers (64KB)
(It is often better for applications to work in chunks fitting in CPU cache.)
compressed from 9149KB to 368KB
data is recovered without loss
The idea being the chunk-based compression is that you don't want to compress a large block of data (that does not fit in cache) in most applications. However, this being said, it does not mean that the example fails to compress the whole thing, it does... It just does so in chunks.
Admittedly, maybe I should better explain it... if so, please say so.
If you think it is wrong, then it is something else, please explain to me why it is wrong.
@lemire example.java in the project looks perfectly fine :
https://github.com/lemire/JavaFastPFOR/blob/1d650e40f6ce3052042871161488971052c9fd32/example.java#L252-L254
But the code from @mokun looks weird, as it calls Arrays.equals over arrays of different length (i.e. it is doomed, hence I really can't see what could be done here before closing the ticket (i.e. I feel some tickets deserve being closed)).
I think that I am going to close this issue. I am always happy to receive criticism and questions, but I need to understand it to be able to process it. In this instance, I do not understand what the issue is.