alievk
Intel Acceleration
I have an AMD rig, so performance is pretty lacking. However, I wonder if this could be compiled to leverage an Intel Movidius stick for acceleration.
https://software.intel.com/content/www/us/en/develop/articles/intel-movidius-neural-compute-stick.html
Perhaps this could be compiled against OpenVINO -- drawing some inspiration from this:
https://www.pyimagesearch.com/2019/04/08/openvino-opencv-and-movidius-ncs-on-the-raspberry-pi/
I'm looking into the feasibility of this and hoping to solicit some feedback. Feel free to delete if this is the wrong place for this kind of thing.
To run the Pytorch model on OpenVINO you would first need to convert it to ONNX. This could require changing some layers if some of them are unsupported in ONNX. This could require some effort but doable for sure.
Hmm... so the pre-existing models won't be directly usable... Is this what you mean? https://michhar.github.io/convert-pytorch-onnx/
https://docs.openvinotoolkit.org/latest/_docs_install_guides_installing_openvino_linux.html
Post training OpenVINO optimization tool kit - https://docs.openvinotoolkit.org/latest/_README.html
https://software.intel.com/content/www/us/en/develop/documentation/oneapi-programming-guide/top/introduction-to-oneapi-programming/oneapi-programming-model-overview.html
https://oneapi-src.github.io/oneDNN/index.html