openvino icon indicating copy to clipboard operation
openvino copied to clipboard

Use array for tracking memory usage instead of map

Open ialbrecht opened this issue 1 year ago • 2 comments

Details:

  • Any additional locking and synchronization on memory allocation might have negative impact on MT execution.
  • std::map has very slow access are requires lock on every access. We can use std::array instead to hold compile time known number of buckets.
  • array container has lower access latency and memory overhead.
  • We might me able to remove mutex lock on stat collection.

ialbrecht avatar Jun 27 '24 20:06 ialbrecht

build_jenkins

rkazants avatar Jun 28 '24 03:06 rkazants

Hi @ialbrecht, any data with concrete numbers on how it can affect MT execution in your case?

rkazants avatar Jun 28 '24 03:06 rkazants

Hi @ialbrecht, any data with concrete numbers on how it can affect MT execution in your case?

We had benchmarks were memory allocation was on the critical performance path and we were looking at all possible ways to optimize it.

ialbrecht avatar Jul 01 '24 14:07 ialbrecht

build_jenkins

vladimir-paramuzov avatar Jul 26 '24 09:07 vladimir-paramuzov