yilong2001 comments

Results 3 comments of


                                            yilong2001

ChatGLM2 lora finetuning 加载 lora 参数：RuntimeError: Expected 4-dimensional input for 4-dimensional weight [3072, 32, 1, 1], but got 3-dimensional input of size [1, 64, 4096] instead

用这种方式加载，也是一样的问题： ```python model = AutoModel.from_pretrained(model_name_or_path, trust_remote_code=True, device_map='auto', torch_dtype=torch.bfloat16) model = PeftModel.from_pretrained(model, peft_model_id) model = model.eval() ```

ChatGLM2 lora finetuning 加载 lora 参数：RuntimeError: Expected 4-dimensional input for 4-dimensional weight [3072, 32, 1, 1], but got 3-dimensional input of size [1, 64, 4096] instead

如果这样加载（先做一次 eval）： ```python tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True) model = AutoModel.from_pretrained(model_name_or_path, trust_remote_code=True, device_map='auto', torch_dtype=torch.bfloat16) model = model.eval() model = PeftModel.from_pretrained(model, peft_model_id) ``` 在这一步会出现如下问题： ```bash ValueError: We need an `offload_dir` to dispatch...

Can this handle classification models, e.g. MNLI?

> @okpatil4u implemented classification here #11 I don't know if they ever made the changes public Here is the bert classification model with ggml: https://github.com/yilong2001/berts.cpp