python_backend icon indicating copy to clipboard operation
python_backend copied to clipboard

WIP Allow Python backend to directly write Numpy arrays to SHM

Open asos-danielbunting opened this issue 2 years ago • 1 comments

asos-danielbunting avatar Jun 24 '23 13:06 asos-danielbunting

Hi @Tabrizian I'm looking at trying to speed up passing a large tensor between a Python BLS model doing preprocessing and a Tensorflow inference model.

As you say the idea is to allocate the buffer and directly write my data into it from the python side and so avoid an extra allocation + copy time. I've run a couple of tests and for my use case this can speed up my inference time by a decent amount eg for a 100000 x 200 float32 tensor the saving was 30ms

asos-danielbunting avatar Jul 03 '23 16:07 asos-danielbunting