gongye19

attack_eval.ieval. how to run this step on gpu??

1

for result in tqdm.tqdm(attack_eval.ieval(correct_samples), total=len(correct_samples)): this step is so slow! how can do this on GPU??

为什么要在配置完又重新把模型dtype设置为fp32

1

model = AutoModelForCausalLM.from_pretrained( args.model_name_or_path, device_map=device_map, load_in_4bit=True, torch_dtype=torch.float16, trust_remote_code=True, quantization_config=BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True, bnb_4bit_quant_type="nf4", llm_int8_threshold=6.0, llm_int8_has_fp16_weight=False, ), ) ...... model = get_peft_model(model, config) model.print_trainable_parameters() model.config.torch_dtype = torch.float32

多节点sft一直卡在这里，微调llama3 8b

1

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction [default0]:[2024-05-01 10:26:58,994] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to cuda (auto detect) [default1]:[2024-05-01 10:26:58,994] [INFO] [real_accelerator.py:191:get_accelerator]...

AttributeError: type object 'AttentionMaskConverter' has no attribute '_ignore_causal_mask_sdpa'

4

集群上多节点sft遇到的问题 huggingface上下载的FlagAlpha/Llama3-Chinese-8B-Instruct llama factory框架

pl hf cl ac 分别是指什么？

2

pl hf cl ac 分别是指什么？

ModuleNotFoundError: No module named 'matchzoo.version'

1

I followed the instructions but still meet this issue