Qwen model issues & embedding and loss has nan

[lylcst]

after a loss backward and optimizer step, then forward the embedding layer output hidden states become inf and loss is nan.

[LMXKO]

+1

[kar9999]

请问是sft阶段还是dpo阶段啊，我在作者的框架下，用sft微调chatglm3 loss也会nan

[lylcst]

请问是sft阶段还是dpo阶段啊，我在作者的框架下，用sft微调chatglm3 loss也会nan

dpo

[akshayraghavan21]

Hi, any update on this? Were you able to fix this issue?

[John-Watson123]

I've got this problem too when using the model Qwen2.5-7B. Python Output: Computing eval metrics: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:20<00:00, 1.26s/it] Generating samples...: 0%| | 0/1 [00:00<?, ?it/s]Both max_new_tokens (=2048) and max_length(=512) seem to have been set. max_new_tokens will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation) Generating samples...: 0%| | 0/1 [00:00<?, ?it/s] Error executing job with overrides: [] Traceback (most recent call last): File "~/direct-preference-optimization/train.py", line 114, in main worker_main(0, 1, config, policy, reference_model) File "~/direct-preference-optimization/train.py", line 44, in worker_main trainer.train() File "~/direct-preference-optimization/trainers.py", line 320, in train policy_samples, reference_samples = self.get_batch_samples(local_eval_batch) File "~/direct-preference-optimization/trainers.py", line 188, in get_batch_samples policy_output = self.policy.generate( File "~/anaconda3/envs/dpo/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "~/anaconda3/envs/dpo/lib/python3.10/site-packages/transformers/generation/utils.py", line 2024, in generate result = self._sample( File "~/anaconda3/envs/dpo/lib/python3.10/site-packages/transformers/generation/utils.py", line 3020, in _sample next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1) RuntimeError: probability tensor contains either inf, nan or element < 0