onnxruntime
onnxruntime copied to clipboard
[Web] Can't create a session
Describe the issue
I'm trying to load a model I've converted from gguf to onnx using optimum-cli and I get this error (Can't create a session).
To reproduce
I'm following the example https://github.com/microsoft/onnxruntime-inference-examples/blob/main/js/quick-start_onnxruntime-web-bundler/main.js with my own local model.
Note that I can run the model successfully in python with:
from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("./public/models/onnx/onnx")
model = ORTModelForCausalLM.from_pretrained("./public/models/onnx/onnx")
inputs = tokenizer("My name is Philipp and I live in Germany.", return_tensors="pt")
gen_tokens = model.generate(**inputs,do_sample=True,temperature=0.9, min_length=20,max_length=20)
response = tokenizer.batch_decode(gen_tokens)
print("Generated text:", response)
Urgency
No response
ONNX Runtime Installation
Released Package
ONNX Runtime Version or Commit ID
1.19.2
Execution Provider
'wasm'/'cpu' (WebAssembly CPU)