steeve
Streaming API
It would be awesome to have streaming compressor/decompressor API as with zlib.compressobj and zlib.decompressobj.
Even more useful would be: being able to set any configuration on the compressor/decompressor objects, like i.e. HC mode and such.
Yes – and specifically, to be able to create or read arbitrarily-large ".lz4" files created by the lz4 command-line. I believe in practice that would mean supporting this: https://docs.google.com/document/d/1cl8N1bmkTdIpPLtnlzbBSFAdUeyNo5fwfHbHU7VRNWY/edit
Can anyone confirm that's what's needed and that it doesn't already exist?
So far I wasn't able to find any lz4 library that supports streaming, and especially for large files it's a requirement.
I checked out the two in that site and a couple more, but no currently none supports streaming/linewise iteration.
Interesting, both these projects announce streaming capabilities for large files, though it could be different from what you are looking for exactly. Little details matter.
The project at github.com/python-lz4 continues this work. We have added frame support, and streaming support is planned for the future.
@Cyan4973 They support the LZ4 streaming (frame?) interface, but they don't do that in a way that it's compatible with Python bytestreams (io.RawIOBase).