streamvbyte
streamvbyte copied to clipboard
lemire
Fast integer compression in C using the StreamVByte codec
Currently, the code is only tested on little-endian hardware. We would need to run tests and to flip byte order (to preserve interoperability) on big-endian hardware.
I have an interest in using streamvbyte with signed types (specifically int16) and have created a Python wrapper which supports this however the overhead is quite large as you would...
The current lookup tables are quite large. Finding a way to substantially reduce their memory usage without adversally affecting performance would be a worthy goal.
Some look-ups could be efficiently replaced by fast instructions such as a [``pdep``](https://software.intel.com/en-us/node/523995) followed by a multiplication and a shift. It is unlikely to be generally faster than a look-up,...
Switching SSE to non-temporal store gives a large speedup (~50% on my i5) in decoding. Non-temporal loads don't seem to make a difference in encoding. Both require 16-byte alignment, so...
Following this PR https://github.com/lemire/streamvbyte/pull/26 we now have code that can use a 0,1,2,4 byte encoding. However, it is basically achieved through pure code duplication. Worse: it does not benefit from...
The generic codec supports both x64 and ARM NEON, however the differential-encoded version is x64 only. It seems like it would be easy to port them over. The Delta function...
* Create a cmake target for proper installation * Update `README.md` with `CMake` integration instructions * Add cmake `find_package(streamvbyte)` check to `Ubuntu 22.04` CI * Set version in cmake according...
For speed, we should have a NEON version of streamvbyte_compressedbytes, see https://github.com/lemire/streamvbyte/pull/57/files
Guess the new Vector API would make it possible to port the project to Java, for instance.
