Text encoder errors in both SD3 dreambooth training scripts

[neuron-party]

Describe the bug

Multiple mismatched argument errors in train_dreambooth_sd3.py when training with train_text_encoder set to True. Ordering/assignment errors in train_dreambooth_lora_sd3.py when training with train_text_encoder set to True.

Reproduction

set --train_text_encoder and all other necessary args appropriately

Logs

For train_dreambooth_sd3.py: encode_prompt got unexpected argument text_input_ids

For train_dreambooth_lora_sd3.py UnboundLocalError: local variable 'text_encoder_one' referenced before assignment

No response

System Info

diffusers from source (bleeding edge)

Who can help?

No response

[sayakpaul]

Possible to share your exact training command?

Cc: @linoytsaban here.

[neuron-party]

@sayakpaul Sure

Essentially the same for both train_dreambooth_lora_sd3.py and train_dreambooth_sd3.py, the only difference is that i specify the rank for the lora script. I train on A100s.

accelerate launch --config_file config_file_path train_dreambooth_lora_sd3.py (or train_dreambooth_sd3.py)
 --mixed_precision=fp16
 --resolution=1024
 --pretrained_model_name_or_path=stabilityai/stable-diffusion-3-medium-diffusers
 --num_validation_images=8
 --validation_epochs=100
 --train_batch_size=2
 --lr_scheduler=cosine
 --checkpointing_steps=200
 --report_to=tensorboard
 --train_text_encoder
 --instance_prompt=xyz
 --instance_data_dir=xyz
 --validation_prompt=xyz
 --max_train_steps=1600
 --rank=64
 --learning_rate=5e-6
 --text_encoder_lr=1e-6
 --output_dir=xyz
 --logging_dir=xyz

[sayakpaul]

@linoytsaban could you take a look here?

[linoytsaban]

hey @neuron-party! just to make sure, you're using only the instance prompt - no custom prompts per image?

[neuron-party]

@linoytsaban Yup

[linoytsaban]

hey @neuron-party! could you please try again with the PR #8755 script? when I ran an experiment with text encoder training & no custom instance prompts I also got an error (that is fixed by #8755) but it was a different one than you got

[neuron-party]

@linoytsaban Sorry for the late reply, I'm running into some function errors on train_dreambooth_sd3.py

TypeError: encode_prompt() got an unexpected keyword argument text_input_ids_list

train_dreambooth_lora_sd3.py seems to be working now though. Thanks for the fixes!

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.