biobert icon indicating copy to clipboard operation
biobert copied to clipboard

Why not filter out the [PAD] tokens when computing loss?

Open zdgithub opened this issue 3 years ago • 0 comments

I found that the losses of the [PAD] tokens were also computed when training the model. But in fact, we should filter out them as https://github.com/kyzhouhzau/BERT-NER/blob/master/BERT_NER.py does. Why didn't you mask the losses of the [PAD] tokens?

zdgithub avatar Mar 03 '22 03:03 zdgithub