intcomp
intcomp copied to clipboard
ronanh
Fast integer compression library
Encoding as uint32 or uint64 slices is a bit cumbersome. We can use byte slices instead without any loss of performance. Go even optimizes the binary.ByteOrder into native operations. Decoding...
A few performance optimisations: * Use array pointers when possible * Eliminate bounds checks * Assigning compressed bytes in a single statement improves the generated code Overall achieves 5-10% improvements...
When small blocks are added to a compressed buffer, the end of the compressed buffer is rewritten to make a bigger block with better compression. This behaviour is not transparent...
When you have an incompressible block that yields the same or more than BitPackingBlockSizeXX+1, you should have a fast uncompressed mode. You have plenty of unused space in your blocks...
> Force creation of blocks at fixed boundaries to enable arbitrary position decoding and reversed iteration The idea of fixed offsets doesn't sound like a great solution to seeking. The...
Is it theoretically possible? Example: ``` comp := intcomp.compress([10,20,30]) // get number by value incomp.Search(comp, 10) // 1 incomp.Search(comp, 20) // 2 incomp.Search(comp, 30) // 3 // find value >=...
