microstream-one
why microsteam occupy 58G main memory?
Environment Details
MicroStream Version:08.01.00-MS-GA JDK version:21.01 OS:Ubuntu 18.04.6 LTS, centos 7 Used frameworks: Spring boot 2.7.8
Describe the bug
it sees the directly buffer is not released for every @fh-ms @fh-ms could you help see this problem? or there is some setting we can do limit the direct memory size. Thank you very much!
java.lang.OutOfMemoryError: Cannot reserve 4096 bytes of direct buffer memory (allocated: 25769800004, limit: 25769803776) at java.base/java.nio.Bits.reserveMemory(Bits.java:178) at java.base/java.nio.DirectByteBuffer.
To Reproduce
user LazyArrayList and write data, remove data by multi threads(200+), the machine has 220G main memory, the jvm heap is set 64G, no other program runs on this machine, the total count of the lazyArrayList item is about 30,000, the total size is about 300M
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
Additional context
Add any other context about the problem here.
At the first glance it seems that the housekeeping is not aggressive enough.
- Check if items removed from the list are no more referenced so that they can be delete by the java GC. If not, the storage will keep the cached.
- You can try to use the ‘adaptive’ StorageHousekeepingController:
final EmbeddedStorageConfigurationBuilder builder = EmbeddedStorageConfigurationBuilder.New();
builder.setHousekeepingAdaptive(true);
...
final EmbeddedStorageFoundation<?> foundation = builder.createEmbeddedStorageFoundation();
final EmbeddedStorageManager storage = foundation.createEmbeddedStorageManager().start();
- Try a manual house keeping configuration
- Decrease
entity-cache-threshold - Decrease
entity-cache-timeout - Increase
housekeeping-time-budgetSee https://docs.microstream.one/manual/storage/configuration/housekeeping.html
At the first glance it seems that the housekeeping is not aggressive enough.
- Check if items removed from the list are no more referenced so that they can be delete by the java GC. If not, the storage will keep the cached.
- You can try to use the ‘adaptive’ StorageHousekeepingController:
final EmbeddedStorageConfigurationBuilder builder = EmbeddedStorageConfigurationBuilder.New(); builder.setHousekeepingAdaptive(true); ... final EmbeddedStorageFoundation<?> foundation = builder.createEmbeddedStorageFoundation(); final EmbeddedStorageManager storage = foundation.createEmbeddedStorageManager().start();
- Try a manual house keeping configuration
- Decrease
entity-cache-threshold- Decrease
entity-cache-timeout- Increase
housekeeping-time-budgetSee https://docs.microstream.one/manual/storage/configuration/housekeeping.html @fh-ms Thanks your reply.
My current setting is like this , Could you give some suggestion？ I used eclipseStore 1.1.0 and java21
var builder = org.eclipse.store.storage.embedded.configuration.types.EmbeddedStorageConfiguration.Builder();
builder.setHousekeepingAdaptive(true);
var foundation = builder
.setStorageDirectory(graphDbPath)
.setChannelCount(1)
.setDataFileMaximumSize(org.eclipse.serializer.configuration.types.ByteSize.New(2,
org.eclipse.serializer.configuration.types.ByteUnit.GB))
.setHousekeepingInterval(Duration.of(5, ChronoUnit.SECONDS))
.setHousekeepingTimeBudget(Duration.of(3, ChronoUnit.SECONDS))
.createEmbeddedStorageFoundation();
foundation.onConnectionFoundation(org.eclipse.serializer.persistence.binary.jdk8.types.BinaryHandlersJDK8::registerJDK8TypeHandlers);
var manager = foundation.createEmbeddedStorageManager().start();
manager.persistenceManager().objectRegistry().ensureCapacity( 1024 * 1024 * 10);
return manager;
I’d try to: decrease the HouseKeepingIntervall, to run it more often (default is once per second) increase the setHousekeepingTimeBudget carefully (default is 10ms) decrease the EntityCacheTimeout (default is 1 Day)
.setHousekeepingInterval(Duration.of(100, ChronoUnit.MILLIS))
.setHousekeepingTimeBudget(Duration.of(50, ChronoUnit.MILLIS))
.setEntityCacheTimeout(Duration.of(1, ChronoUnit.MINUTES))
With this setup housekeeping will be done every 100ms for a 50 ms max and clear cache items that are older than one minute.