BlenderSourceTools
BlenderSourceTools copied to clipboard
Optimizing validate_element in datamodel.py for faster loading of large dmx files
I had a scenario where I was trying to load a relatively large dmx file using datamodel.py and it was taking a very long time. I noticed that validate_element is using an O(n) lookup for every element added during load, so I added some sets to convert this to O(1), unless validation fails.
Time comparison
In the case of my dmx file (SFM session, ~57MB), these were the load time before and after the change:
- before: 1hr+ (I left it running for 1 hour and it's still not finished)
- after: 10 sec
Let me know if you'd like me to make any adjustments.
This is certainly a problem, thank you for identifying it. However these changes cause problems when modifying a datamodel because we never remove items from either set. I'll have to think more about a solution.