python-rocksdb
python-rocksdb copied to clipboard
How to optimize the speed when I import a large data into pyrocksdb?
When I import a large data into rocksdb, the speed is very slow. How can I optimize the speed?thank you!
Replace the hard disk with an SSD, and provide your rocksdb status log, search for some optimization solutions about delayed write and write blocking,If you are importing existing data to rocksdb, you can try to create an sst file to import
If you are doing an initial bulk load, insert the data in order, sorted by key. Sorting the data up front makes for much less work when loading.
You can also create the SST files yourself- https://github.com/facebook/rocksdb/wiki/Creating-and-Ingesting-SST-files