llama-cpp-python

llama-cpp-python copied to clipboard

Running server gives error when using huggingface model

3

850 return issubclass(cls.__origin__, self.__origin__) 851 if not isinstance(cls, _GenericAlias): --> 852 return issubclass(cls, self.__origin__) 853 return super().__subclasscheck__(cls) TypeError: issubclass() arg 1 must be a class The manual package works with...

djaffer

Getting `Illegal instruction (core dumped)` when running an openblas made install

4

Making the latest llama.cpp repo with `make LLAMA_OPENBLAS=1` works as it should. But running `LLAMA_OPENBLAS=1 pip install --force-reinstall --no-cache-dir --no-index --find-links . llama-cpp-python` (running on an offline server, its the...

Bloob-beep

Currently this comes up as an AssertionError which leads to a lot of confusion

abetlen

Add `/models/{model}` endpoint

1

abetlen

* Add a feature with unlimited max_tokens that is enabled when max_tokens

jm12138

Fix unicode decoding error

1

Certain tokens in the vocabulary cannot be decoded to valid utf-8, I'm actually not sure if this is because they represent partial utf codepoints, but in any case they cause...

abetlen

[Windows] "Failed building wheel for llama-cpp-python"

2

Edit : For now i've installed the wheel from "https://github.com/Loufe/llama-cpp-python/blob/main/wheels/llama_cpp_python-0.1.26-cp310-cp310-win_amd64.whl". The installation of the wheel works. So everything is fine for me. Got things working also in WSL with no...

kerbi

abetlen

Investigate model aliasing

3

Allow the user to alias their local models to OpenAI model names as many tools have those hard-coded. This may cause unexpected issues with tokenization mismatches.

abetlen

abetlen