Ouyanmei

icon location My name is Ouyanmei, majoring in cyberspace security at Guangdong University of Foreign Studies

Results 4 comments of Ouyanmei

phonetic预训练细节

> 论文里面说使用带错误的训练数据预训练phonetic Encoder， 但代码里面好像是用的纠正后的数据，不知道我有没有理解错，恳请解惑 ![image](https://user-images.githubusercontent.com/33060143/174463691-6dc3e499-3a81-452b-8737-ad97b814ef2b.png) 我也有这个疑问，你好，请问解决了吗

The actor constantly generates ['</s>'] or ['<|endoftext|></s>'] after 200 steps in RLHF with hybrid engine disabled

你好，请问你解决了吗

zero3 and enable hybrid engine are not suitable for llama2, how to solve it?

请问解决了吗

Actor loss nan and Resizing model embedding

Due to the vocabulary size of the GPT-2 124M model being 50257, resizing the model's embedding layer dimensions may result in new embeddings that exceed the original vocabulary range. This...