孙先森

icon indicating a website link https://www.zhihu.com/people/sunshubing icon indicating an email [email protected]

icon location Peking icon location NLP with deep learning. Mainly focus on text classification, text similarity，intention recognition, QA & task oriented dialogue.

Results 2 comments of 孙先森

数据处理中re.findall('##[\u4E00-\u9FA5]')作用

> 请问预训练中数据处理whole word mask 中这一行有什么作用，我发现如果去掉这行后效果会显著下降 > > https://github.com/brightmart/roberta_zh/blob/13f7849f0cb0e11573e032acddb35b83b096224e/create_pretraining_data.py#L526 这不就是取除了##的中文部分token吗

在pytorch模型上做post train

> 请问找到解决办法了吗？🌝🌝 https://github.com/sliderSun/chinese_pytorch_pretrain_lm