Qixiang Mo
Qixiang Mo
同，我也发现了这个问题，很神奇
@NohTow Thanks for your reply! Sorry for not making it clear. Predicting the label of each word in the whole sentence is just like sequence labeling, that's why am using...
> @NohTow Thanks for your reply! Sorry for not making it clear. Predicting the label of each word in the whole sentence is just like sequence labeling, that's why am...
> From the code I am reading, you are loading the weights correctly when doing > > ``` > self.encoder = ModernBertModel.from_pretrained( > model_path, config=config, attn_implementation="flash_attention_2") > ``` > >...
@NohTow Hi there, I think I found the problem. When I loaded the model using ModernBertPreTrainedModel, and I tried to customize a new classifier layer, I found that the initial...
@NohTow Thank you for your answer. This problem is not related to `self.init_weights()`. Maybe the `self.classifier` is not initialized correctly from `.from_pretrained()` ?