OpenLogReplicator icon indicating copy to clipboard operation
OpenLogReplicator copied to clipboard

LOB: use general memory buffers for storing LOB data

Open bersler opened this issue 2 years ago • 3 comments

The current implementation does not use memory buffers to allocate LOB data. Instead heap pool is used. This leads to a situation that the total memory usage can not be limited and information presented about memory usage does not include memory allocated for LOBs. It would be best to actually use also memory buffers for LOB.

bersler avatar Mar 19 '23 00:03 bersler

This feature is complicated to implement.. If there is a demand from users more focus could be given here.

bersler avatar Mar 19 '23 00:03 bersler

The lack of memory buffer usage for LOB data allocation, relying instead on the heap pool, significantly affects the management of large data loads. This method leads to uncontrolled memory usage increases and lacks transparency in memory consumption metrics. Introducing memory buffers for LOB data would allow for better resource management, crucial for high-volume data environments.

Alternatively, capturing LOB data directly from the source database using a control connection could be explored, but this might be more complex and suitable as a separate feature.

Given the importance of efficient large data load management, prioritizing memory management improvements for LOBs is essential. This would greatly enhance system performance and scalability.

marlon-costa-dc avatar Jan 28 '24 18:01 marlon-costa-dc

So far this not an issue for any of OpenLogReplicator users that I am aware of. Thus the priority is low.

bersler avatar Jan 28 '24 22:01 bersler