IGNITE-16538 Enable RocksDB to control memory usage better with stalling write if memory usage exceeds limit

[Myasuka]

https://issues.apache.org/jira/browse/IGNITE-16538

[Myasuka]

cc @ibessonov

[Myasuka]

I checked the guide of how to contribute, why I cannot view the CI test results?

[ibessonov]

Hi @Myasuka, CI report can be found here: https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunAllTests?focusLine=0&branch=pull%2F655&mode=builds#all-projects It's also available on this page in "checks" section. I see a problem in logs and I don't quite get it, the process is stuck:

   java.lang.Thread.State: RUNNABLE
	at org.rocksdb.RocksDB.write0(Native Method)
	at org.rocksdb.RocksDB.write(RocksDB.java:1784)
	at org.apache.ignite.internal.storage.rocksdb.RocksDbPartitionStorage.removeAllExact(RocksDbPartitionStorage.java:287)
	at org.apache.ignite.internal.storage.AbstractPartitionStorageTest.testRemoveAllExact(AbstractPartitionStorageTest.java:543)

Do you now any possible reasons for this? Main branch works just fine.

EDIT: Oh, I think I get it, you can't batch write more than it is configured in write buffer size, otherwise it'll stall in the middle.

Is there any way to preemptively detect such situation? I would not want large transactions to break the cluster. Should we just avoid using batches when it's possible? Looking forward to your reply, thank you!

[Myasuka]

@ibessonov Thanks for your guid for the CI result.

The test hangs as we make the write buffer manager could stall, I need to invistigate the reason.