Bert-Chinese-Text-Classification-Pytorch icon indicating copy to clipboard operation
Bert-Chinese-Text-Classification-Pytorch copied to clipboard
verified publisher icon 649453932

AttributeError: 'NoneType' object has no attribute 'tokenize'

Open Cgetier520990 opened this issue 1 year ago • 4 comments

Traceback (most recent call last): File "run.py", line 28, in train_data, dev_data, test_data = build_dataset(config) File "D:\PYTHON_PROGRAMME\Bert-Chinese-Text-Classification\utils.py", line 36, in build_dataset train = load_dataset(config.train_path, config.pad_size) File "D:\PYTHON_PROGRAMME\Bert-Chinese-Text-Classification\utils.py", line 20, in load_dataset token = config.tokenizer.tokenize(content) AttributeError: 'NoneType' object has no attribute 'tokenize' 这个报错怎么解决？

Cgetier520990 avatar Jul 24 '24 02:07 Cgetier520990

我也出现这个错误，你解决了吗

wangling6666 avatar Jul 26 '24 12:07 wangling6666

没有诶，你现在解决了吗？

Cgetier520990 avatar Aug 08 '24 08:08 Cgetier520990

@wangling6666 @Cgetier520990 我刚刚拉完代码也有这个问题，从huggingface把vocab.txt下载后放到模型文件夹就行

CNMBE avatar Aug 14 '24 06:08 CNMBE

是因为从网上拉下来的文件名称不一致，要保证名称为vocab.txt并且在bert_pretrian的目录下

TiezMind avatar Dec 07 '24 12:12 TiezMind