huggingface
Improve control net block index for sd3
What does this PR do?
The layer configuration for the Control Net in Stable Diffusion 3 models must adhere to the rule that the total number of layers in the SD3 model should be a multiple of the Control Net's layer count.
For SD3.5, which has 38 layers, the Control Net can only have three possible options: 2, 19, or 38. This leads to inefficiencies in the setup.
Who can review?
@sayakpaul @yiyixuxu @DN6
This is a very simple PR, but for some reason, the tests all failed for wired reasons such as:
Unable to find self-hosted runner group: 'aws-general-8-plus'.
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
@sayakpaul
There is an existing test for the pipeline of SD3 control net, I leveraged that. The layer number for the control net changed from 1 to 3, the number of layer of the transformer (4) is no longer the multiples of the number of layers of control net. This will test the code changes of this PR.
