Carl Silva

i posted an issue on gradio. no idea how to get it to stop doing this.

try this: # bitsandbytes debugging in case no GPU error: pip uninstall bitsandbytes && pip install bitsandbytes python -m bitsandbytes && python -m torch.utils.collect_env if you still get the error,...

yup got it working.

nope: python convert-to-gptq.py -m ./gpt-cmd -o ./gpt-cmd-gptq CUDA extension not installed. CUDA extension not installed. /home/silvacarl/.local/lib/python3.8/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will...

Docker file seems to be out of date? get this error: $ docker build --no-cache --force-rm -t autogptq . => ERROR [9/9] RUN pip install -vvv . 1.9s ------ >...

i have seen the exact same thing. this technique appears to not be as fast as faster-whisper implementation. in fact, it seems to be about two times slower. so i...

thx checking that out

super cool, will check it out when merged. just fyi, we are benchmarking these: SinanAkkoyun/oasst-sft-7-llama-30b databricks/dolly-v2-12b Aeala/GPT4-x-AlpacaDente2-30b NousResearch/gpt4-x-vicuna-13b LLMs/Stable-Vicuna-13B nomic-ai/gpt4all-13b-snoozy togethercomputer/GPT-NeoXT-Chat-Base-20B mosaicml/mpt-7b-instruct mosaicml/mpt-7b-chat TheBloke/koala-13B-HF EleutherAI/pythia-12b mosaicml/mpt-1b-redpajama-200b-dolly stabilityai/stablelm-tuned-alpha-7b TheBloke/wizardLM-7B-HF samwit/koala-7b couchpotato888/alpaca13b...

ok, so this is new: python server.py --verbose --model-menu --trust-remote-code --load-in-8bit INFO:Gradio HTTP request redirected to localhost :) WARNING:trust_remote_code is enabled. This is dangerous. bin /home/silvacarl/.local/lib/python3.8/site-packages/bitsandbytes/libbitsandbytes_cpu.so INFO:Loading TheBloke_stable-vicuna-13B-HF... ╭─────────────────────────────── Traceback...

how can i disable auto-devices, like --auto-devices False?