Fail in running Llama-3.2-1B

[ycdfwzy]

Hello, I have my first try to run swiftLLM with Llama-3.2-1B, but get the following error: Python version 3.9.20 torch version 2.4.0

[interestingLSY]

It seems that Llama-3.2-1B does not have lm_head.weight. Instead, it uses model.embed_tokens.weight as lm_head.weight.

For a quick fix, you may try to modify this line: https://github.com/interestingLSY/swiftLLM/blob/af7a5589fdac7b2d8b080ed34f2be706f20724a0/swiftllm/worker/weight.py#L154 . You may try to change lm_head.weight to model.embed_tokens.weight.

[ycdfwzy]

Thanks for your reply. I got another error after the modification

[interestingLSY]

Could you try to print rope_scaling_factor? It should be a int, but it seems that it is a dict from the error message

[zuston]

I met the similar problems.

Could you try to print rope_scaling_factor? It should be a int, but it seems that it is a dict from the error message

Yes. The detailed structure info of rope_scaling_factor as follows

{'factor': 32.0, 'high_freq_factor': 4.0, 'low_freq_factor': 1.0, 'original_max_position_embeddings': 8192, 'rope_type': 'llama3'}

It looks we could only get the factor in the rope_scaling_factor . Right? @interestingLSY

[zuston]

Oh, I think this is not simply to get this value directly, just like this issue: https://github.com/alibaba/Pai-Megatron-Patch/issues/361