InvokeAI icon indicating copy to clipboard operation
InvokeAI copied to clipboard

Add support for FLUX ControlNet models (XLabs and InstantX)

Open RyanJDick opened this issue 1 year ago • 0 comments

Summary

Add support for FLUX ControlNet models (XLabs and InstantX).

QA Instructions

  • [ ] SD1 and SDXL ControlNets, since the ModelLoaderRegistry calls were changed.
  • [x] Single Xlabs controlnet
  • [x] Single InstantX union controlnet
  • [x] Single InstantX controlnet
  • [ ] Single Shakker Labs Union controlnet
  • [x] Multiple controlnets
  • [x] Weight, start, end params all work as expected
  • [x] Can be used with image-to-image and inpainting.
  • [x] Clear error message if no VAE is passed when using InstantX controlnet.
  • [x] Install InstantX ControlNet in diffusers format from HF repo (InstantX/FLUX.1-dev-Controlnet-Union)
  • [ ] Test all FLUX ControlNet starter models

Merge Plan

No special instructions.

Checklist

  • [x] The PR has a short but descriptive title, suitable for a changelog
  • [x] Tests added / updated (if applicable)
  • [ ] Documentation added / updated (if applicable)

RyanJDick avatar Oct 08 '24 21:10 RyanJDick