table-transformer icon indicating copy to clipboard operation
table-transformer copied to clipboard

Memory leak in using table detection

Open Srivishnu27feb opened this issue 2 years ago • 1 comments

Hi Team, I have a pdf of 20 pages and I am using pdf2image library to convert to images and passing each image for detection using threads and I could see the memory gets piled up and is not getting deallocated unless the entire application is exited... I tried debugging the code just when using table detection or table structure det the memory gets piledup.. in my flask application i am loading the model once and reusing in my function for infering each pages of pdf and also tried gc.collect() and del the variables but no luck. Currently i am using the huggingface implemtation.. Is there any work around that could help in release the memory?

Srivishnu27feb avatar Apr 10 '23 03:04 Srivishnu27feb

I am facing the similar issue, memory gets allocated during requests and later the memory doesn't get deallocated which is making the application to crash eventually. I performed memory profiling also while sending in multiple requests, the application is at a constant memory usage prior the request stream and after the stream. But the memory that the application is holding on to has increased (please check the attached image.) plot_predict_tatr

balajiChundi avatar Jan 11 '24 09:01 balajiChundi