Shikhar Singh comments

Repositories
Issues
Comments

Results 3 comments of


                                            Shikhar Singh

LlamaTokenizer has no `pad` token, leading to failure during batch-tokenization

@basujindal My general understanding for bs > 1, we need to pad during finetuning. However, in pretraining the input text is set to max-length -- you can think of a...

RuntimeError: self and mat2 must have the same dtype

Was facing this error with GPT-2 as well, with peft==0.3, but upgrading to 0.4 resolved it. (`fan_in_fan_out=True`)

RuntimeError: self and mat2 must have the same dtype

@NNDEV1 Sure! Although I am using Bits&Bytes for quantization. ```python import torch from transformers import BitsAndBytesConfig, AutoModelForCausalLM from peft import LoraConfig, get_peft_model # BnB (4-bit) bnb_cfg = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type='nf4',...