ValueError: Attempting to unscale FP16 gradients when fully finetunning on LLM without LoRA?

[dragen1860]

dear all: when i disable the lora and set whole Mistral LLm weights requires_grad=True, it gives me the error:

ValueError: Attempting to unscale FP16 gradients

anyone give some tips?

[Qnancy]

I meet the same problem

[Qnancy]

have you solved it?