afgonczol comments

Repositories
Issues
Comments

Results 5 comments of


                                            afgonczol

Fresh install and fails to load. "raise RuntimeError('Error(s) in loading state_dict"

I get the same error. It seems to happen when using pre_layer > 0.

Fresh install and fails to load. "raise RuntimeError('Error(s) in loading state_dict"

As Ph0rkk0z mentioned, it's due to a mismatch between the gptq used with oobabooga and the gptq used to quantize the model. I've found that 4bit maintains versions of the...

stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors fails to load

Same issue here.

stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors fails to load

> Looks like it's caused by using pre_layer. No solution yet that I'm aware of.

Short answers are generated. I need long ones.

> Get extra RAM (at last 32GB) and use the 30B model. It provides quite long replies and has excellent reasoning skills. Can you share what settings and prompts you're...