Improve performance of graph operations.
Three separate sources of slowness have been identified:
-
Early in the process when data is being gathered, profiling shows Hibernate eating a lot of memory. Caches may need configuration and later versions of Hibernate may make that easier.
GraphTraversalwill tend not to be querying the same objects repeatedly. -
In reviewing the data gathered via Hibernate,
GraphPolicyRuleeats much CPU time in rule-matching. The internal implementation of the class could do non-trivial static analysis when initialized, compiling the list of rules to a decision tree of checks, branches, changes, making the subsequent matching much faster. -
In actioning the model changes, PostgreSQL can spend much time in managing the
_reindexing_requiredtable. Perhaps migration to https://github.com/glencoesoftware/omero-es can somehow help.
This issue has been mentioned on Image.sc Forum. There might be relevant details there:
https://forum.image.sc/t/omero-storage-reports/41819/2
non-trivial static analysis when initialized
This could also spot opportunities to reorder rules to reduce number of passes in cases where outcome would remain unchanged. However, there are probably not big wins as the user can already pluck low-hanging fruit, e.g., listing transitions to RELEVANT before transitions from that to IS_NOT_LAST.