Performance of diffusers

[StrugglingForBetter]

I trained a model with diffusers. But I found the testing performance is not stable for the trained model. Sometimes, it can generate high-quality images, but sometimes, the images does not make sense or may be weird or distorted. What factors may affect the permformance? How can I improve the model performance? I need a model which can generate high-quality images every time.

I also found larger num_inference_steps won't make the performance better, sometimes even worse.

@patil-suraj @pcuenca @patrickvonplaten

[patrickvonplaten]

Hey @StrugglingForBetter,

Could you try to be a bit more specific? What model did you train? What example script did you use? What data did you use? ...

Thanks!

Patrick

[StrugglingForBetter]

Hey @StrugglingForBetter,

Could you try to be a bit more specific? What model did you train? What example script did you use? What data did you use? ...

Thanks!

Patrick

Hey, @patrickvonplaten @pcuenca @patil-suraj

I trained a text_to_image model using diffusers. Scipt is just like: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image.py

I use our own private data. Images are products images, and text is Chinese desciption.

It can generate high-quality images for some samples but not so good for the others.

Is there any way to improve the performance?

[patil-suraj]

It depends on the dataset and hyperparameters. Also, it's unlikely to have a model that always generates high-quality images. I suggest using a bigger and better-quality dataset and playing around with the hyperparameters (lr, training epochs etc).

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.