comfyanonymous
[Feature Request] Support for bundled LoRA/embedding files
Onetrainer recently added an option to simultaneously train embeddings and LoRAs (I think this is called pivotal tuning?) and bundle them into a single safetensor file. I believe the format for this was actually created a while ago (see the first link). I experimented a bit with training using this method and got good results, and it seems like it can be an effective method for multi-concept LoRAs.
In A1111/web UI these work as expected, just include "lora:loraname:1 embeddingname" in the prompt. However I can't find a way to activate the embedding in a bundled file in Stable Swarm or the ComfyUI backend, only the LoRA. Separate embedding files work when loaded with embed:embeddingname but bundled files are a lot more convenient for large multi-concept LoRAs. Would support for this be something that can be implemented by ComfyUI?
Ideally, bundled embeddings should only be "accessible" when the associated LoRA is active, which is how web UI appears to work from my testing. Or there may be a better way to avoid conflicts between identically named embeddings in multiple bundle files.
Some links for reference: Original web UI pull request for the bundle format OneTrainer pull request Relevant section of the OneTrainer code
As requested by mcmonkey, here's a couple example bundles. "bundle_example.safetensors" is a SD 1.5 lora, "bundle_example_sdxl.safetensors" is an SDXL lora. You can see the format, and how it corresponds to the existing 1.5/XL/etc. standards, just by enumerating the keys in the file.
It would be good to get this supported; it's available in both A1111 and SD.Next, and people are out there creating loras with bundled embeddings and they simply do not work in ComfyUI. models.zip
Relevant commits from that example file and discord comments after https://github.com/comfyanonymous/ComfyUI/commit/1c08bf35b49879115dedd8ec6bc92d9e8d8fd871 https://github.com/comfyanonymous/ComfyUI/commit/e1c528196ef77e8c69b67d96dc909b8ccb776007
