Qnancy comments

Results 8 comments of


                                            Qnancy

ValueError: Attempting to unscale FP16 gradients when fully finetunning on LLM without LoRA?

I meet the same problem

ValueError: Attempting to unscale FP16 gradients when fully finetunning on LLM without LoRA?

have you solved it?

How to get the stage 2 checkpoint path for 3_sft.sh

> same question... do you get the access? No one has replied to me yet...

UserWarning: None of the inputs have requires_grad=True. Gradients will be None

> It's okay~ 感谢回复！我已经使用之前说的情况训练得到了ckpt_latest.pth，但在inference的时候出现了'San francisco, california, usa. june 2019: a man cleaning a couch. \n\nSan francisco, california, usa. june 2019: a man cleaning a couch.‘类似的乱码，可能是什么情况呢

UserWarning: None of the inputs have requires_grad=True. Gradients will be None

> @Qnancy 如果您使用的是vicuna的语言模型，请检查一下是否是下载的delta模型并已经完成了权重的转换。转换的过程您可以参考https://github.com/OpenGVLab/Ask-Anything/blob/main/video_chat/README_CN.md#running-%E4%BD%BF%E7%94%A8%E6%96%B9%E6%B3%95 感谢回复~我使用的是mistrial，且在训练前使用videochat2_mistral_7b_stage3.pth进行inference是完全没有问题的，将checkpoint_path变成训练得到的ckpt_latest.pth就输出了乱码