bert.cpp icon indicating copy to clipboard operation
bert.cpp copied to clipboard
verified publisher icon skeskinen

Bugfix: missing hparam `type_vocab_size`

Open FFengIll opened this issue 2 years ago • 3 comments

  • type_vocab_size is also a hparam (can not use const as 2).
  • so does the converter.

FFengIll avatar Sep 14 '23 10:09 FFengIll

Here is some of the model upon bert DO NOT use type_vocab_size=2 but type_vocab_size=1 (like e5).

https://huggingface.co/intfloat/multilingual-e5-base/blob/main/config.json#L25

FFengIll avatar Sep 14 '23 10:09 FFengIll

Hi, This seems like a good change.

But surely bert.cpp:358 also needs to be changed? Where the hparams are read from the model file

skeskinen avatar Sep 18 '23 13:09 skeskinen

Hi, This seems like a good change.

But surely bert.cpp:358 also needs to be changed? Where the hparams are read from the model file

sure, I will add it.

FFengIll avatar Sep 19 '23 01:09 FFengIll