Diffusers

A Unity package to run pretrained diffusion models with Unity Sentis

OpenUPM · Documentation (coming soon) · Feedback/Questions

About

This is essentially a port of Hugging Face’s diffusers library.

It is still very early though, so as of today only a limited number of pipelines and schedulers are supported (see below).

Roadmap

Some things that might be worked on next are:

• [x] add/fix classifier-free guidance
• [ ] support more models
  • [x] SD 1.5
  • [x] SD 2.1
  • [x] SDXL
  • [x] SD-Turbo
  • [x] SDXL-Turbo
  • [ ] LoRA models
• [ ] more scheduler implementations
  • [x] PNDM
  • [x] DDIM
  • [x] EulerDiscrete
  • [x] EulerAncestralDiscrete
  • [ ] LCM
  • [ ] DDPM
  • [ ] KDPM2Discrete
  • [ ] KDPM2AncestralDiscrete
• [ ] more pipeline/model types
  • [x] txt2img
  • [x] img2img,
  • [ ] inpaint,
  • [ ] upscale,
  • [ ] depth2img
  • [ ] marigold
• [x] async pipeline methods (sliced inference)
• [ ] support multiple images per prompt
• [ ] write some documentation

Dependencies

• com.doji.transformers
• com.doji.pngcs