apache
HDDS-11404. Make RocksDB all parameters configurable.
What changes were proposed in this pull request?
Make RocksDB all parameters configurable.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-11404
How was this patch tested?
unit tests and local test environment, and i will provide some test results from my local test environment
Additional notes:
- We can use different ini files to customize the rocksDB configuration of corresponding services (such as OM, SCM, DN). For example, you can create a file such as om.db.ini to configure OM's rocksDB, scm.db.ini to configure SCM's rocksDB, and container.db.ini to configure DN's rocksDB
- OM, SCM, and DN were tested separately, and the results were in line with expectations
for OM
om.db.ini file:
OPTION file:
for SCM
scm.db.ini file:
OPTION file:
for DN
container.db.ini file:
OPTION file:
@jojochuang @szetszwo @adoroszlai Could you help review this PR? Thanks.
@weimingdiit instead of wrapping every config via Ozone, why not let Ozone specify the RockDB config file https://github.com/facebook/rocksdb/blob/5b40c0c074f16dae6e1d60a3c5d7aa3e7d63fdc7/include/rocksdb/utilities/options_util.h#L56 This way, we do not need to keep managing RocksDB Config value additions or deprecation.
@kerneltime I agree. I made the same suggestion in #7117 with a pointer to the RocksDB docs. The approach taken here is not sustainable to maintain and also less intuitive than having Ozone work with RocksDB's config file exactly how RocksDB documents it.
This DBProfile abstraction is old and not really used as far as I know. It might be better to just get rid of it. We can instead directly provide example RocksDB config files for different use cases. This will be much easier to follow than hiding the tuning inside the Java code.
@errose28 @kerneltime This is a very good idea. I looked at the relevant code and felt that directly supporting INI files might be a better choice. Thank you very much for your suggestion. I will modify the code to fully support INI files. By the way, I think this issue https://github.com/apache/ozone/pull/7117 should be closed
@weimingdiit Do we have mechanism to control service specific usecase, like if OM, SCM and DN have different configuration, like WAL manual flush may need disable only for OM ?
Do we have mechanism to control service specific usecase, like if OM, SCM and DN have different configuration, like WAL manual flush may need disable only for OM ?
Probably we could have configuration files for each Role . By default all of these can point to a base config but also provide an option to add overrides for each role. Similar to ozone.metadata.dirs where each process can define its own metadata dir but if not defined , we can fallback to the common config
Do we have mechanism to control service specific usecase, like if OM, SCM and DN have different configuration, like WAL manual flush may need disable only for OM ?
Probably we could have configuration files for each Role . By default all of these can point to a base config but also provide an option to add overrides for each role. Similar to ozone.metadata.dirs where each process can define its own metadata dir but if not defined , we can fallback to the common config
@sumitagrawl @sadanand48 Yes, this may be our ultimate goal.
DBConfigFromFile already supports configuring DBOptions from a ini file. But it doesn't configure column family options.
I also wonder if opening up rocksdb options and make all of them configurable is the right thing to do. It's possible to shoot in the foot. Worst case is causing corruption (not sure if it's possible). Most of the projects based on RocksDB opens up a few selected rocksdb options.
Also note there are existing Ozone configuration properties that overrides the RocksDB options loaded from ini file. A proper documentation would be needed to help clarify the interactions of these properties.
If I understand correctly, this is continued (with different approach) in #8173.
