Cheng Qian comments

Results 5 comments of


                                            Cheng Qian

预训练第一阶段，需要冻结原版LLaMA词表的embedding吗

请问训练整个词表是否会影响原本的embedding？冻结原版LLaMA的embedding会不会效果更好呢？请问您有没有做过这个实验，或者怎么看待这个问题呢？感谢您的解答！

Distinguish between bold and non-bold Fonts

Do you find any methods to extract these bold texts such as subtitles? Thank you!

预训练llama 13b报错 NotImplementedError: Cannot copy out of meta tensor; no data!

torch==1.13.1 transformers==4.28.1 还是会报这个错

Some weights of GPTNeoXForCausalLM were not initialized from the model checkpoint

I met exactly the same issue except that I am using llama. Did you find the correct way to load the trained rm model?

The engine generated by each build has different results for the same input.

> Can you provide more details, i.e the cmds, which can help us reproduce this issue? I encountered same issue. I trtllm-build 2 times with everything identical, but the inference...