onnxruntime icon indicating copy to clipboard operation
onnxruntime copied to clipboard

[Web] Can't create a session

Open djannot opened this issue 1 year ago • 0 comments

Describe the issue

I'm trying to load a model I've converted from gguf to onnx using optimum-cli and I get this error (Can't create a session).

To reproduce

I'm following the example https://github.com/microsoft/onnxruntime-inference-examples/blob/main/js/quick-start_onnxruntime-web-bundler/main.js with my own local model.

Note that I can run the model successfully in python with:

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("./public/models/onnx/onnx")
model = ORTModelForCausalLM.from_pretrained("./public/models/onnx/onnx")

inputs = tokenizer("My name is Philipp and I live in Germany.", return_tensors="pt")

gen_tokens = model.generate(**inputs,do_sample=True,temperature=0.9, min_length=20,max_length=20)
response = tokenizer.batch_decode(gen_tokens)
print("Generated text:", response)

Urgency

No response

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.19.2

Execution Provider

'wasm'/'cpu' (WebAssembly CPU)

djannot avatar Oct 17 '24 18:10 djannot