rethinkdb
Add a compressed output format for rethinkdb export / import
Is your feature request related to a problem? Please describe. Dumping large databases / tables involves writing very large files to the disk, is expensive in terms of disk io.
Describe the solution you'd like Add a new compressed output format json.gz to rethinkdb export. Enable on the fly streaming compression of data files using zlib.
This is an interesting (and useful) feature that surely deserves to be implemented.
As of now, we're in the middle of a revamp of this repo, because the project is flooded with old code and bad mixes of python 2 and 3. We're already working on it, in a separate repo, where we're cleaning up the messy code and start supporting only python 3 since the EOL of python 2.
Our first goal is to not break any existing code. We will then start adding features as well, like the one requested in this issue.
This is something I'm experimenting with at the moment. Does it make sense to open a PR with a proposed implementation to get some feedback?
If there's already the implementation, we could try to make room for it in the new repo (temporarily private since early WIP).
You can use the python-2-to-3 branch (or master if already started working from that one). Please try to keep your code as separate as possible from the existing code, so we can extract it easily and then port it to the new codebase.
Cool, I opened a PR against master for now, that's what I was working on. If it's a go I can switch to python-2-to-3.
I checked the PR rapidly, but it looks really neat. Thanks!
It will need a proper code review and we will then merge it into the new repo (as soon as we're done with the basis of it). I would keep the PR open until then, if you don't mind, do you?