pipe.scheduler.sigmas change after forward call

[neuron-party]

Describe the bug

I noticed this issue when trying to manually modify the sigmas in my pipeline scheduler. Despite changing the values in this tensor, the sigmas tensor shrinks to the number of inference steps after one inference instance. Subsequent calls no longer change the values of the sigmas.

Reproduction

pipe.scheduler.sigmas = pipe.scheduler.sigmas * 4 # manually modifying sigmas _ = pipe(..., num_inference_steps = N) print(pipe.scheduler.sigmas) # len is N + 1, values do not reflect the manual modifications and no longer change in subsequent calls

Logs

No response

System Info

diffusers==0.27.2

Who can help?

No response

[yiyixuxu]

hi! if you are looking to use custom sigmas, you can pass it down as sigmas from your pipeline see doc https://huggingface.co/docs/diffusers/main/en/using-diffusers/schedulers#custom-timestep-schedules and this PR https://github.com/huggingface/diffusers/pull/7817

you will need to install diffusers from the source and use the most recent updated version

[yiyixuxu]

also simgas change after forward call is intended behavior because we reset the timesteps based on the num_inference_steps argument

[neuron-party]

Great thanks! I noticed this functionality still isn't supported for SDXL ControlNets, specifically diffusers.StableDiffusionXLControlNet pipeline, and I was wondering if you will be adding this functionality for this pipeline? Thanks!

[yiyixuxu]

yes - do you want to open a PR to help us add this to the Controlnet pipeline? if not we can ask other members of the community for help too:)

[neuron-party]

PR: https://github.com/huggingface/diffusers/pull/7913

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.