Runtume error on a finetuned model

[realtimeriddle]

I'm getting an error when I run pplm with a gpt2 model I finetuned with the language modeling example from the huggingface transformers repo.

run_pplm.py -B /path/to/BOW.txt --pretrained_model=/user/FindtunedModelOut --cond_text="potato" --num_samples=20 --length=150 --stepsize=0.03 --num_iterations=3 --window_length=5 --gamma=1.5 --gm_scale=0.95 --kl_scale=0.01 --colorama --verbosity=regular --sample

and I get the error:

Traceback (most recent call last): File "/pythonProjects/transformerTest/venv/PPLM/run_pplm.py", line 936, in run_pplm_example(**vars(args)) File "/pythonProjects/transformerTest/venv/PPLM/run_pplm.py", line 768, in run_pplm_example unpert_gen_tok_text, pert_gen_tok_texts, _, _ = full_text_generation( File "/pythonProjects/transformerTest/venv/PPLM/run_pplm.py", line 472, in full_text_generation pert_gen_tok_text, discrim_loss, loss_in_time = generate_text_pplm( File "/pythonProjects/transformerTest/venv/PPLM/run_pplm.py", line 584, in generate_text_pplm pert_past, _, grad_norms, loss_this_iter = perturb_past( File "/pythonProjects/transformerTest/venv/PPLM/run_pplm.py", line 213, in perturb_past bow_logits = torch.mm(probs, torch.t(one_hot_bow)) RuntimeError: mat1 dim 1 must match mat2 dim 0

I'm not sure if I screwed up the finetuning or pplm, but the model does generate text with the run_generation example, and if I just change the model to gpt2 pplm runs on the bag of words. Anyone know how to fix this error, or what I am doing wrong? Thanks.

Edit: The problem seems to have to do with the special tokens I added.

[dathath]

Can you elaborate what "gpt2 pplm runs on the bag of words" means? I don't know if I fully follow what is causing the problem. I haven't seen this error in this context before.

[iedr]

Hi, I have the same problem; I think the issue lies with added tokens in the finetuning process... Any ideas how to solve this? E.g. I added X number of tokens in the finetuning process. These tokens are added to added_tokens.json, but this addition is not reflected when I run the pplm example. The tokenizer still has the 50257 number of tokens. This error happened in the perturb_past function.

[realtimeriddle]

Can you elaborate what "gpt2 pplm runs on the bag of words" means? I don't know if I fully follow what is causing the problem. I haven't seen this error in this context before.

I meant that pplm runs without error with the pretrained gpt2 model, without my fine-tuneing, using a bag of words. I was trying to relay that pplm works with other models and that the model, not the bag of words, was causing the problem. If I remove the two lines of code that I use to add my special tokens to huggingface's language modeling example, pplm does run.

special_tokens_dict = {'bos_token': '<SOT>', 'eos_token': '<EOT>', 'pad_token': '<PAD>'}
num_added_toks = tokenizer.add_special_tokens(special_tokens_dict)