elixir-nx
Upgrade to ort 2.0.0-rc.10?
Hi!
I was wondering if there was any work going on to upgrade this library to use ort 2.0.0-rc.10?
I have a project where I am going to use the new RTX 5090 and this is on the Blackwell architecture and require CUDA Compute version 120.
To be able to use this it looks like I need to use a newer version of ONNXRuntime then the current 1.16 that is used in the ortex library at the moment..
Error while trying to run with CUDA as EP:
Non-zero status code returned while running QuickGelu node. Name:'/model.0/act/Mul/QuickGeluFusion/' Status Message: CUDA error cudaErrorNoKernelImageForDevice:no kernel image is available for execution on the device
I tried just using a newer version of ONNXRuntime library by replacing the *.so files but then I get the following error:
{:load_failed, ~c"Failed to load NIF library: 'libonnxruntime.so: version "VERS_1.19.0" not found (required by /opt/maskon/cameratest/_build/dev/lib/ortex/priv/native/libortex.so)'"}}
This indicated to me that the libraries are incompatible and I started to rewrite ortex to use the newest ort version as this uses a newer version of ONNXRuntime.
Only problem is that my rust programming capabilities are somewhat limited and I have now reached a point in the rewrite where I just dont understand where to go on anymore..
This is as far as I have gotten and it might not even be correct, but its my try!
https://github.com/m0rt3nlund/ortex/tree/upgrade
Anyone else who want to chime in and help get this working work be amazing!
Thank you!
We've been waiting on ort to get a full 2.0 release so that it's API stable before upgrading ortex, otherwise the work we put in to upgrade may be a waste (this has already happened once since it's been in 2.0-rc). There's a decent amount of work to upgrade last I checked.
We've also considered decoupling from ort and going directly to libonnxruntime, but also it's a bigger lift. It would alleviate this issue, and with ort being in RC for almost 2 years now it's left us many versions behind on onnxruntime (as you've noticed).
Hi :) Thanks for the reply! I noticed ort write on their frontpage that the api should now be more or less stable and they promote usage. Thats why i got motivated to try to rewrite it. Made PR for you with my current work, which compiles and works now :) I forgot to run the tests but i have it running in my own environment now in a stress-test on real data using CUDA as EP on a RTX 5090 card.