RWKVSharp icon indicating copy to clipboard operation
RWKVSharp copied to clipboard

Does ONNX models used in this project supports CUDA only( crashes on CPU)

Open JohnMasen opened this issue 2 years ago • 3 comments

HI, I'm trying to run your models with CPU, the application crashes while loading the model. To verify if these ONNX models could be loaded by CPU, I created a simple test app like this.

using Microsoft.ML.OnnxRuntime;
using Onnx;
SessionOptions options = new();
options.AppendExecutionProvider_CPU();
InferenceSession session = new InferenceSession(@"D:\Models\RWKV-4-Raven-3B-v11-Eng49-Chn49-Jpn1-Other1-20230429-ctx4096_32_2560_16.onnx", options);
Console.WriteLine("Done");
Console.ReadLine();

the application crashes with following exception

System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'

Could you please kindly suggest if any thing I did wrong or the model bin file is designed for CUDA only?

JohnMasen avatar May 04 '23 16:05 JohnMasen

Which onnx nuget package are you using? The CPU does not need to install cuda when using this'Microsoft.ML.OnnxRuntime '

imxcstar avatar May 05 '23 18:05 imxcstar

Microsoft.ML.OnnxRuntime v1.14.1

JohnMasen avatar May 06 '23 05:05 JohnMasen

This is not clear. Perhaps the configuration environment is different somewhere. 😂

imxcstar avatar May 07 '23 11:05 imxcstar