[Core] refactor embeddings.py so that it becomes a nice module.
What does this PR do?
The embeddings.py script is becoming long and bloated.
This PR breaks the script meaningfully so that it can be supplemented as a module.
- Creates a
embeddings_utils.pyto have common methods shared across the board. - Moves the IP Adapter embeddings blocks to a dedicated
embeddings/ip_adapter.pt. - Does the same for PixArt-related embeddings, too.
This should be very much backwards-preserving.
Once I have an initial approval, I will make sure all the embedding classes are documented, too.
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
@yiyixuxu and @DN6 did you have a chance to think through this? :)
I'm cool to divide them up a little bit by "type":
- positional embeddings is a pretty big category on its own,
- timesteps
- text/image: we only have one text projection embedding so ideally just combine them; they are all simply projection layers
- other: gligen bounding box, label embedding
- combined
Yeah that sounds reasonable to me. I will let @DN6 also comment here and I will take a stab :)
Agree to breaking up by type as well. @yiyixuxu's suggestions make sense to me.
Closing because of brutal merge conflicts. Let's jam on https://github.com/huggingface/diffusers/pull/8722, instead.