Support MuLan, a plug-and-play language adapter to adapt existing diffusion model for up to 110+ languages without additional training

[Zeqiang-Lai]

Model/Pipeline/Scheduler description

# pip install mulankit
from diffusers import StableDiffusionPipeline
+ import mulankit

pipe = StableDiffusionPipeline.from_pretrained('Lykon/dreamshaper-8')
+ pipe = mulankit.transform(pipe, 'mulanai/mulan-lang-adapter::sd15_aesthetic.pth')
image = pipe('一只蓝色的🐶 in the 바다').images[0]

一只蓝色的 🐶 in the 바다 (Dreamshaper-8)	レゴシュワルツェネッガー (SDXL-lightning)	一只可爱的猫头鹰 (MVDream)	海浪风景 (AnimateDiff)

MuLan supports

• Base models: Stable Diffusion 1.5, 2.1, XL, Pixart-Alpha/Sigma.
• Downstream models: ControlNet, LCM, LoRA, finetuned models and etc.
• Video models: AnimateDiff.
• 3D models: MVDream.

Open source status

• [X] The model implementation is available.
• [X] The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

https://github.com/mulanai/MuLan https://huggingface.co/mulanai/mulan-lang-adapter