lost in dream

请教一下大家lora和原模型合并的一个问题

1

如下是一段在https://github.com/lvwerra/trl/blob/fc468e0f3582de1aacd071fceb24265c619a8ef5/examples/stack_llama/scripts/merge_peft_adapter.py中截取的代码 # Load the Lora model model = PeftModel.from_pretrained(model, script_args.adapter_model_name) model.eval() key_list = [key for key, _ in model.base_model.model.named_modules() if "lora" not in key] for key in key_list: parent, target,...

关于UniLM

1

想请问下 使用了UniLM的方案是通过 哪部分代码体现的啊

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

5

▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ Traceback (most recent call last) ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ ▒ /ChatGLM-6B/ChatGLM_math/chatglm_maths/t10_toy_trl_train_ppo.py:215 in ▒ ▒ ▒ ▒ ▒ ▒ 212 ▒ # get model response ▒ ▒ 213 ▒ # print(query_tensor) ▒ ▒...