geray
geray
I met the same problem，what is the final solution?
OK,I solved it by remake qaq
I also meet same problem.
The reference of documentation is that [llama_server](https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md). And I try to rewrtire gguf.py like follow: ```python import logging import time import requests from requests.exceptions import RequestException from tqdm import tqdm...
> What model here is being evaluated? I will try to look into this soon. Qwen1.8B I discovered that the original script gguf.py, when matched with llama-cpp-python to start the...
> hi, I met the same problem as yours, have you solved this problem? try to use llama-cpp-python replace llama.cpp start server
> In GGUF it should determine based on the offsets which tokens are part of the continuation and which are not (the `while` loop in the screenshot skips any context...
I meet same error at [#1637](https://github.com/EleutherAI/lm-evaluation-harness/issues/1637)
Thank you for your interest in this topic. If you would like to proceed, please feel free to send an email to [email protected]. Your understanding is correct, and we can...
fixed by https://github.com/apecloud/kubeblocks-addons/pull/341