ValueError: The tokenizer already contains the token ... Please pass a different `placeholder_token` that is not already in the tokenizer.

[OzzyD]

Just a quick question regarding tokens:

I have a model trained on a new token and wish to add more detail and add extra run cycles to it. I get the above advice/error

How do I go about fixing this issue, or am I best to retrain from scratch?

[patrickvonplaten]

Hey @OzzyD,

Could you post a reproducible code snippet as well as your environment information? :-)

[Kenoubi]

@OzzyD I assume you are running https://github.com/huggingface/diffusers/blob/main/examples/textual_inversion/textual_inversion.py (or the Colab notebook with mostly the same code.) You can just comment out that check, although in that case you should also comment out token_embeds[placeholder_token_id] = token_embeds[initializer_token_id] as this would essentially restart the training from scratch anyway.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.