fgdfgfthgr-fox
fgdfgfthgr-fox
The issue seems to be resolved in the last pull, at least I am not experiencing it anymore. Likely from commit #de6a09d
You should try with python 3.9.
> The q4_x files output from ggml are not compatible with llama.cpp? It seems so currently.
Can you post your detailed system specs?
> Are you using the latest gradio version? > > ``` > pip install -r requirements.txt --upgrade > ``` Updated to the latest gradio, still face the same issue. My...
Updated to the latest webui version as well as dependences again. Tried launching in both terminal and Pycharm, still the same issue.
Updated to gradio 3.24.0 version, still no improvement. Maybe it's something to do with web browser? I was using firefox.
Seems fixed indeed!
I managed to make FlexGen work for Galactica-1.3b model by changing opt_config.py, flex_opt.py and tokenizer_config.json. @oobabooga 's Webui can successfully load the model and generate text using it. Vram use...
> @fgdfgfthgr-fox can you create a fork of https://github.com/FMInference/FlexGen with your changes? @oobabooga https://github.com/fgdfgfthgr-fox/FlexGen---galactica-support Is this what you want?