Enable Volume Generation from Concrete Labelmaps in MAISI ControlNet
Is your feature request related to a problem? Please describe. Image generation with MAISI
Describe the solution you'd like
Afaiu, currently, MAISI can "only" generate volumes conditionally on controlnet_cond. These are relative size changes of certain anatomical labels. But it cannot generate a volume from a concrete labelmap that is given.
Describe alternatives you've considered
In ControlNet's forward() method, I did not see an alternative that would allow embedding a concrete labelmap. The Diffusion Model requires embeddings from ControlNet, and cannot accept concrete labelmaps either.
Additional context Figure 2 in the MAISI paper also seems to indicate that ControlNet can only accept "task-specific conditions", and "Diffusion Model" depends on ControlNet embeddings.