abetlen
Getting `Illegal instruction (core dumped)` when running an openblas made install
Making the latest llama.cpp repo with make LLAMA_OPENBLAS=1 works as it should.
But running LLAMA_OPENBLAS=1 pip install --force-reinstall --no-cache-dir --no-index --find-links . llama-cpp-python (running on an offline server, its the tar.gz you get from pip download)
produces an Illegal instruction (core dumped) when you try to generate words from it.
Can you check out and test this llama.cpp commit with OPENBLAS (this is what v0.1.32 is based on) https://github.com/ggerganov/llama.cpp/tree/684da25926e5c505f725b4f10b5485b218fa1fc7 to confirm, also compare to the latest llama.cpp
Sorry my bad, I did not think things through. As for the commit you sent it works. What i didn't do was I forgot to test on the actual llama-cpp-python repo itself, it works for llama-cpp-python as well.
My error appeared when I was using langchain, it does not like chain.running the llm. Illegal instruction (core dumped)
llm = LlamaCpp(model_path="/opt/gpt/model/ggml-vicuna-7b-4bit-rev1.bin", n_ctx=2048, max_tokens=100, n_batch=512, stop=["\n"])
chain = load_qa_chain(llm, chain_type="stuff")
answer = chain.run(input_documents=found_docs, question=query)
@Bloob-beep but without the chain ie llm(prompt) doesn't give this error? Very strange
Correct me if i'm misunderstanding, but yes I think so.
It can load the llm, load the chain. But as soon as I try to run the chain it doesn't work. Works on non-BLAS versions though.
