ChatGLM-6B icon indicating copy to clipboard operation
ChatGLM-6B copied to clipboard

[BUG/Help] <lm_head.weight权重>

Open 447428054 opened this issue 3 years ago • 0 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

Current Behavior

为什么模型加载时缺少该权重:lm_head.weight

@mojianhao @AdamBear @jsl9208 @cjld

不在model.named_pamaters也不在model.named_buffers,这样的权重经过skip_init只有随机初始化,不进行梯度更新吗

Expected Behavior

为什么模型加载时缺少该权重:lm_head.weight

不在model.named_pamaters也不在model.named_buffers,这样的权重经过skip_init只有随机初始化,不进行梯度更新吗

Steps To Reproduce

  1. 加载模型

Environment

- OS:linux
- Python:3.8
- Transformers:4.27
- PyTorch:1.10
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :

Anything else?

No response

447428054 avatar Apr 17 '23 13:04 447428054