Sarthak comments

Results 5 comments of


                                            Sarthak

token prefix in roberta model?

Thanks now I can load my model. But I get this error now, what am I missing ? ![FireShot Capture 001 - analysis-Copy1 - Jupyter Notebook - localhost](https://user-images.githubusercontent.com/17705073/153586307-dc7ea7ed-1980-4c7e-9c52-881c76a69656.png)

Using run_mlm.py to pretrain a roberta base model from scratch outputs do not include <bos> or <eos> tokens

Is there any update about this issue, I'm facing the same error? @ArthurZucker

Using run_mlm.py to pretrain a roberta base model from scratch outputs do not include <bos> or <eos> tokens

![Screenshot 2023-04-04 at 5 13 48 PM](https://user-images.githubusercontent.com/17705073/229838857-88c6ff16-4c95-4178-ba2c-6b93d69c701f.png) Training without special tokens also doesn't work, not sure what is the reason then

Using run_mlm.py to pretrain a roberta base model from scratch outputs do not include <bos> or <eos> tokens

I trained it with return_special_tokens_mask=False, but only for 3 epochs (is it possible that when I train it fully it's able to learn) ?

Using run_mlm.py to pretrain a roberta base model from scratch outputs do not include <bos> or <eos> tokens

I trained the model for 75 epochs, still and tokens are not appearing