off-heap memory used by oakMap can not be released

[fujian-zfj]

I use oakMap instead of concurrentSkipListMap in my project, while I find something strange. When I no longer use an oakMap, java heap memory used by this oakMap is released, but off-heap memory does not seems to be released.

I use NMT to track the use of off-heap memory, and I find something strange.

ggggggggg

The heap memory occupied by current process is very small, but through the linux system command "top" found that it occupies huge memory. And I analyze the memory distribution in heap and find that heap memory used by the current process is very small. By the way, therer is no OakMap object in heap memory.

[fujian-zfj]

private final OakMapBuilder<Long, AckData> oakMapBuilder; private final OakMap<Long, AckData> unAckMessagesMap;

public OffHeapMessagePool() {
    this.oakMapBuilder = new OakMapBuilder<>(new OakLongComparator(), new OakLongSerializer(),
        new OakAckDataSerializer(), Long.MIN_VALUE).setMemoryCapacity(MEBIBYTE);
    this.unAckMessagesMap = oakMapBuilder.buildOrderedMap();
}

public boolean addToUnAckMessageMap(long deliveryTag, AckData ackData) {
    return unAckMessagesMap.putIfAbsent(deliveryTag, ackData) == null ? true : false;
}

public boolean removeFromUnAckMessagesMap(long deliveryTag) {
    return unAckMessagesMap.remove(deliveryTag) != null ? true : false;
}

public void clean() {
    if (unAckMessagesMap != null ) {
        try {
            unAckMessagesMap.close();
            //System.gc();
        } catch (Exception e){

        }
    }

}

[liran-funaro]

Oak has a cached memory pool. It reuses the memory for other Oak instances. After closing Oak the memory is not released back to the OS, but new Oak instances will reuse this memory. Can you tell us more about your use case? Do you have a case where you don't create new Oak instances but need to release the memory back to the OS?

[fujian-zfj]

I create 100 OakMap. One thread keeps calling oakMap.putIfAbsent(key, value), and the other thread keeps calling oakMap.get(key) and oakMap.remove(key). During this period, no new OakMap will be created. A few days later, I found that my process is down due to outOfMemroy

[fujian-zfj]

[fujian-zfj]

Yes, I will not create new Oak Instances and need to release the memory back to the OS

[fujian-zfj]

I use MAT(Memory Analyzer Tool）to anlayze the memory usage of my process, and I find this:

[sanastas]

Hi @fujian-zfj ,

Thanks for your interest in Oak and sorry for late reply. The issue that you see is because currently keys in the OakMap are not released and with the time going keys are consuming the memory. It was written that way because until now OakMap was used for relatively short periods of time and keys are usually not big.

However, we already have PR#188 where we are adding the new memory manager and the ability to release keys to OakMap. So it is mater of a week or so. If you need it really urgently, may be you would like to try OakHash (not ordered) instead of OakMap (ordered). In OakHash keys are released in time of the mapping remove. Are you using public release or current master?

[fujian-zfj]

I use current master. And what I need is OakMap(ConcurrentSkipListMap) instead of OakHash.

I am looking forward to the new memory manager which has the ability to release keys to OakMap.

If possible, I hope it can be launched as soon as possible, and I will continue to follow up on this issue.