Steven Stulga

:handshake: I'm all about that 32GB of HBM2 I've installed exllama via pip using the command line that you provided, but I'm still seeing the ImportError related to hipblasGetStream ```...

``` (textgen) myuser@mymachine:~/text-generation-webui$ echo $PATH /opt/rocm/bin:/home/myuser/miniconda3/envs/textgen/bin:/home/myuser/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin ``` Full output: ``` (textgen) myuser@mymachine:~/text-generation-webui$ python server.py --notebook --model airoboros-65B-gpt4-1.4-GPTQ --loader exllama --gpu-split 20,20 --listen --api 2023-07-13 10:14:19 INFO:Loading airoboros-65B-gpt4-1.4-GPTQ... 2023-07-13 10:14:20 WARNING:Exllama...

Do you think that it is potentially an environment/system issue with the ROCm installation and not an exllama issue? I've tried running with both the pip module installed and exllama...

I've been having mostly success running GPTQ single-GPU by following that rentry.co guide already. I say "mostly success" because some models output no tokens, gibberish, or some error; but other...

They are both in physical x16 slots but one is running in PCIE Gen3x16 mode and the other is unfortunately running PCIE Gen3x4 mode at the moment.

I've got a 4600G for APU, so it detects 3 GPUs but rocm-smi does not play super nicely with the APU. I did install every use-case from amdgpu installer before...

We could swap so that we'd both have a match! Ha ha :stuck_out_tongue: Interestingly, one of my MI60s is 1x8-pin PCIe power connector + 1x6-pin PCIe power connector, and the...

Out of curiosity, how are you cooling your MI60s? I am cooling using 1 80mm fan per card, rigged up with 3D printed fan shrouds so that they force air...

> Can you try hiding your APU? I only had problems trying to use mine. > > `HIP_VISIBLE_DEVICES=0,1 ROCR_VISIBLE_DEVICES=0,1` I already have these environment variables set. I've run with them...

> What's the output of `ldd torch/lib/libtorch_hip.so | grep hipblas`? I have been procrastinating reformatting, so I can still tell you this! No output. Here is without grep ``` (textgen)...