Zihui Wu

icon company Xidian University icon location Xian China icon location I am currently a Ph.D. student at Xidian University. My research interests lie in the security of large language models.

Results 5 comments of Zihui Wu

The loss doesn't decrease when using multi nodes.

Hey guys, I have adjusted some code of the forward function in class NT_Xent and now it can work, but I just found the multi-gpu performance is mush worse than...

The loss doesn't decrease when using multi nodes.

> Hey guys, I have adjusted some code of the forward function in class NT_Xent and now it can work, but I just found the multi-gpu performance is mush worse...

Inconsistent Inference Results After Saving and Loading LoRA Model

Same issue, and when I want to solve the problem ``` Error generating response: 'Qwen2ForCausalLM' object has no attribute 'load_lora' ``` by using: ``` from unsloth_zoo.vllm_utils import load_lora .... output...

Inconsistent Inference Results After Saving and Loading LoRA Model

> not sure if it helps, but i also tried this in above notebook and it reproduces the adapter outputs exactly :) Hi, can you share your versions of transformers...

Inconsistent Inference Results After Saving and Loading LoRA Model

This code works for me: ``` from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel import torch import argparse device = "cuda" if torch.cuda.is_available() else "cpu" # Load model and...