ImportError: cannot import name 'BertWordPieceTokenizer'

[stellaywu]

tried installed from pip and source, both returned ImportError: cannot import name 'BertWordPieceTokenizer' on from fast_bert.data_cls

Please help!

[mrjgamble]

I have the same issue. I've tried using 1.6.1 and 1.6.0, but then run into other problems during the fit of a multi-label model.

[dhruv-sharma]

This issue may be due to the version of the tokenizers module. If so then you would need to first install Rust and then install the tokenizers module.

Assuming you have a bash shell and are within your env (actual or venv development environment), you can try the following:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh source $HOME/.cargo/env pip install tokenizers

More logs would help though.

[propower1712]

I solved the issue by adding those two lines befort the import :

import torch

camembert = torch.hub.load('pytorch/fairseq', 'camembert.v0')

[Shane-Neeley]

I had this and then ran pip install --upgrade tokenizers and got past it. Though I'm still nowhere near getting it to work yet ha