xd2333

Results 6 comments of xd2333

Qwen-72B-Chat conversion script does not treat <|im_start|> and <|im_end|> correctly.

in my case, [PAD151645] [PAD151644] [PAD151643] shows in output(Qwen-14B-Chat q4k)

ZeRO3 LoRA微调Qwen1.5-32B-Chat内存一开始不高，后面突然爆内存

类似问题，恢复检查点后显存占用正常

different inference result

> You're correct! It seems like `max_seq_length`'s default of 4096 is auto scaling TinyLlama, causing bad outputs - I'll fix this asap - thanks for the report! Hi unslothai, thx...

【提示】transformers>=4.43.0小显存训练时不加入以下参数容易导致显存累积，直至爆显存/OOM

> > ### Reminder > > > > * [x] I have read the README and searched the existing issues. > > > > ### System Info > > transformers>=4.43.0...

【提示】transformers>=4.43.0小显存训练时不加入以下参数容易导致显存累积，直至爆显存/OOM

> > ### Reminder > > > > * [x] I have read the README and searched the existing issues. > > > > ### System Info > > transformers>=4.43.0...

RuntimeError: Given groups=1, weight of size [5120, 36, 1, 2, 2], expected input[1, 16, 1, 78, 78] to have 36 channels, but got 16 channels instead

for anyone saw this with Wan2.2-I2V-A14B, try use `arch: 'wan22_14b_i2v'`