Multiple prompts

[tralala87]

Model/Pipeline/Scheduler description

Hi.

How to use: from diffusers import FlaxStableDiffusionPipeline

pipeline, params = FlaxStableDiffusionPipeline.from_pretrained( "runwayml/stable-diffusion-v1-5", revision="bf16", dtype=jax.numpy.bfloat16 )

to generate multiple prompts instead of single one (prompt = "a photo of an astronaut riding a horse on mars")?

Open source status

• [ ] The model implementation is available
• [ ] The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

https://github.com/huggingface/diffusers#text-to-image-generation-with-stable-diffusion

[averad]

Which of the following are you attempting to do?

1. Generate multiple images using different prompts.
   • Example: Image 1 - Prompt "Portrait of a Cat, field of flowers, golden hour" Image 2 - "Painting of a Warship, battle, wartime, desktop wallpaper"
2. Generate one image using multiple prompts.
   • Example: prompt = Portrait of a Cat, field of flowers, golden hour

[tralala87]

Generate one image using multiple prompts. Example: prompt = Portrait of a Cat, field of flowers, golden hour

[pcuenca]

Hi @tralala87, I'm not sure I'm following. If you want to use the JAX implementation of diffusers to generate multiple images at once (for different prompts), I'd recommend you follow the steps in this section of our blog post and try it out in the associated colab notebook.

If it's something else, please reopen and do let us know :) Thanks for writing!

[tralala87]

Thank you @pcuenca for your answer. I want to use all 8 tpu machines to produce 100.000 different images from 100.000 prompts and 100.000 seeds, 1 picture per 1 prompt with 1 seed. And not to have images joined into one big image. As I understand, in this section of your blog post, one can only make one picture per 1 tpu machine and then all the images are joined into one single big image. I do not want that.

P.S.: I can't reopen this thread.

[pcuenca]

The images are joined together in the image_grid, just to visualize them in the notebook. If you don't want that, then you need to remove that line and do something else. For example, to save them to disk you'd do something like this:

for i, image in enumerate(images):
    image.save(f"image_{i}.png")

Or course, you need to make sure that you have enough disk space to save them all, and that the filenames you choose are unique so they don't get overwritten if you run the cell multiple times.

[tralala87]

I can't generate more than 8 images (8 prompts), I get "ValueError: cannot reshape array of size 6930 into shape (8,newaxis,77)" when running "prompt_ids = shard(prompt_ids)"

[tralala87]

Any ideas how to solve this? Much appreciated.

[pcuenca]

Hi @tralala87, the number of prompts you supply must be a multiple of 8. This is because that's the number of devices in a TPU computer, as you mentioned in one of your comments above. shard will distribute the prompts among all the available devices, but it won't know how to do it if they are not divisible by 8.