krita-ai-diffusion icon indicating copy to clipboard operation
krita-ai-diffusion copied to clipboard

[ enhancement ] Marigold depth estimator

Open Pirog17000 opened this issue 2 years ago • 3 comments

Would be nice to have a few options to select from for the depth estimation algorithms, specifically new Marigold as it catches a LOT of details https://huggingface.co/spaces/toshas/marigold

Additionally, Zoe if possible. Current depth estimator is fast but outputs are far worse than they could be :/

And again let me tell that the plugin is a best thing right now for diffusion, thank you for this!

Pirog17000 avatar Dec 18 '23 04:12 Pirog17000

I'm having a look at https://github.com/kijai/ComfyUI-Marigold every now and then. It's still a bit rough around the edges, also it's very heavy weight (download size, vram reqs, execution speed). Results are nice though...

Acly avatar Dec 23 '23 11:12 Acly

Apologies if this belongs in a separate issue, but if Marigold isn't currently an option, are other depth model options still being considered? LeReS with boost enabled and matching the resolution to the image height also yields really good results, already comes with stock ComfyUI, and defaults to a tiling method so VRAM shouldn't be a problem.

VFX-Software avatar Mar 31 '24 01:03 VFX-Software

Changed to depth-anything in the last update. It's similar to the one before though, better in some cases.

I tested LeReS too, it catches a LOT more detail, the depth maps look pretty good. In my scenarios it really didn't translate to good results with depth CN though, images contained a lot of non-sensical details and garbage that would have to be cleaned up. They didn't match the input image well at all.

I'm sure it highly depends on the scenario, but it didn't seem like a good general purpose solution.

Acly avatar Mar 31 '24 08:03 Acly