diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard
Published 3 months ago • verified publisher icon huggingface

Adding TextualInversionLoaderMixin for the controlnet_inpaint_sd_xl pipeline

Open jschoormans opened this issue 1 year ago • 8 comments

What does this PR do?

The pipeline controlnet_inpaint_sd_xl pipeline contained the textual_inversion loading logic, but did not load the TextualInversionLoaderMixin yet.

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

jschoormans avatar Mar 12 '24 10:03 jschoormans

Thanks for the PR. Could you share some code and results where this is beneficial?

sayakpaul avatar Mar 12 '24 11:03 sayakpaul

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

HuggingFaceDocBuilderDev avatar Mar 12 '24 11:03 HuggingFaceDocBuilderDev

Hi Sayak,

When using this pipeline with a textual inversion (and without this commit):

self.pipe.load_textual_inversion()

I have this error:

AttributeError: 'StableDiffusionXLControlNetInpaintPipeline' object has no attribute 'load_textual_inversion'

Adding TextualInversionMixIn fixes this - and this seems in line with the other controlnet and sdxl pipelines.

jschoormans avatar Mar 12 '24 11:03 jschoormans

Yes, that is fine. I was curious see the results with that i.e., Textual Inversion + SDXL ControlNet Inpainting (with your fix)

sayakpaul avatar Mar 12 '24 11:03 sayakpaul

can we fix the tests?

yiyixuxu avatar Mar 19 '24 19:03 yiyixuxu

Hi yiyixuxu, I think there was a formatting error, (extra newline) but I am not sure. I have made a new commit which fixes that. I don't really know what else would be wrong. I use this pipeline to do inpainting with loras I have trained, and for that I also add the textual inversion part. It seems to be the only pipeline that was missing this part.

jschoormans avatar Apr 02 '24 10:04 jschoormans

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

github-actions[bot] avatar Apr 26 '24 15:04 github-actions[bot]

hi: can you run make fix-copies and make style?

yiyixuxu avatar Apr 26 '24 20:04 yiyixuxu

looks like make style made changes to files that you did not touch in this PR can you update the quality dependency? pip install -e ".[quality]"

yiyixuxu avatar Apr 29 '24 09:04 yiyixuxu

hmm I tried to checkout the previous commit, did the pip install and ran the make again.

jschoormans avatar Apr 29 '24 09:04 jschoormans

@jschoormans thanks for your contribution! and sorry for the inconvenience

yiyixuxu avatar Apr 29 '24 19:04 yiyixuxu

Thank you!!!

On Mon, 29 Apr 2024 at 21:01, YiYi Xu @.***> wrote:

@jschoormans https://github.com/jschoormans thanks for your contribution! and sorry for the inconvenience

— Reply to this email directly, view it on GitHub https://github.com/huggingface/diffusers/pull/7288#issuecomment-2083455811, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACGUCHWSWOIYFXSJLAFST6DY72KKFAVCNFSM6AAAAABER6WAP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBTGQ2TKOBRGE . You are receiving this because you were mentioned.Message ID: @.***>

jschoormans avatar Apr 29 '24 19:04 jschoormans