OverflowError: cannot fit 'int' into an index-sized integer

[zws98]

when I use the tokenizer, the error occurs: Info:File "/home/z/anaconda3/envs/t2v/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 3737, in _pad encoded_inputs["attention_mask"] = encoded_inputs["attention_mask"] + [0] * difference !:OverflowError: cannot fit 'int' into an index-sized integer

I check the value of "difference" = 1000000000000000019884624838647

[Sakura-xing]

same problems

[Tokymin]

Me too

[heandrea]

"Replace the files in the tokenizer folder with the files of the relevant model type."将tokenizer文件夹下的文件换一下，换成相关类型模型的文件

[my-littledarling]

检查一下名称对不对 check out the name of files