BinaryView Memory Leak in DebuggerController When a ReportCollection Tab is Closed Last
- open file
- open report widget (current_function.request_debug_report("mlil"))
- close the tab/file for that view
- close the report widget tab
- get_memory_usage_info()
This is how the relevant events unfold:
- when we close the tab for the view itself, the debugger actually correctly detects the closing of the last tab that belongs to the view, and destroys the controller for it
- The report widget automatically becomes the current tab
- The report widget has a reference to the binary view, and SetCurrentBinaryView is called with it
- The Python console receives it, and constructs a new DebuggerController using that view
- When the report widget is closed,
NotificationListener::OnBeforeCloseFileis not called, so the debugger does not get a chance to know about - Memory leak
I do not know what is a proper fix for this because:
- The main windows seems to intentionally NOT call
OnBeforeCloseFilefor widget like report widget, settings widget. Which I think is a wise choice - One might argue that the closing of the view tab is NOT technically the last tab of the view (the report widget is still open and it has a reference to the view). But that does not help, since even if we do not destroy the old controller when that happens, we still would not be able to know the reprot tab is finally being closed at all.
To summarize, the debugger relies on OnBeforeCloseFile to know when a file is closed and then cleanup the controller for it. Now the report widget is the last tab that gets closed, and OnBeforeCloseFile is NOT called for it.
I also believe that this also affects the database saving logic in the very same way, since the way the debugger detects the "last open tab" of the file is the same as how main window does it. If we modify the view, and then open the report widget, we will be asked to save the database when the view tab is closed. However, you can still make changes to the view using the Python console, and the main window will not ask you to save it
One option to solve this is to parent any report widgets to the view tab. When the last view tab is closed, all other report tabs with that view are closed first.
I am bumping this since this is not really a regression -- it has been there for a while and we happen to discover it recently