Generation preview

[ambocclusion]

Is it possible to get previews of an in-progress generation, similar to what is shown in the web ui, using comfyscript? Ideally I'd like some form of stream I can process per-step.

[Chaoses-Ib]

TaskQueue._watch() currently doesn't handle previews. I'll add callbacks for them later.

[Chaoses-Ib]

v0.5.0a1 is released. You can get previews like this:

from PIL import Image

# Prevent displaying previews
# <0.5.0
# queue.watch_display(False, False, False)
queue.watch_display(False)

with Workflow() as wf:
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    conditioning = CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,', clip)
    conditioning2 = CLIPTextEncode('text, watermark', clip)
    latent = EmptyLatentImage(512, 512, 1)
    latent = KSampler(model, 12, 20, 8, 'euler', 'normal', conditioning, conditioning2, latent, 1)
    image = VAEDecode(latent, vae)
    SaveImage(image, 'ComfyUI')

def preview(task: Task, node_id: str, image: Image.Image):
    # 'KSampler.0'
    print(node_id)
wf.task.add_preview_callback(preview)

A stream form (sync/async iter) is also possible. But I'm not sure if it‘s worth the effort, as one can get similar results with a callback and a wait.

[ambocclusion]

Holy crap, you rock! Works perfectly!

[Chaoses-Ib]

There are still some things left to do, like reporting progress and preview together, and fixing Jupyter Notebook display issues.

[ambocclusion]

Ah sorry

[ambocclusion]

https://github.com/Chaoses-Ib/ComfyScript/assets/10687655/57d92290-eefb-49b2-80e8-ceefd76425c1

Here are the results of your preview work! Again, I'm really grateful for your work on Comfyscript! Thank you

[Chaoses-Ib]

@madriss-mojo

Is there a way to get the output of other types of nodes that have finished execution. For example if I have several SaveImage or VHS_VideoCombine could we get their output file while the prompt is still in progress ? Either using this or via the ComfyUI API ? Thanks

For SaveImage:

from PIL import Image

with Workflow() as wf:
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    conditioning = CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,', clip)
    conditioning2 = CLIPTextEncode('text, watermark', clip)
    latent = EmptyLatentImage(512, 512, 1)
    latent = KSampler(model, 12, 20, 8, 'euler', 'normal', conditioning, conditioning2, latent, 1)
    image = VAEDecode(latent, vae)
    image = SaveImage(image, 'ComfyUI')

# or: await image
image_batch = image.wait()

# Get the first image
# or: await image_batch.get(0)
image: Image.Image = image_batch[0]
display(image)

# Get all images in the batch
images: list[Image.Image] = image_batch.wait()

For VHS_VideoCombine, see https://github.com/Chaoses-Ib/ComfyScript/issues/22#issuecomment-1971187462.

[ghostsquad]

v0.5.0a1 is released. You can get previews like this:

from PIL import Image

# Prevent displaying previews
queue.watch_display(False, False, False)

with Workflow() as wf:
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    conditioning = CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,', clip)
    conditioning2 = CLIPTextEncode('text, watermark', clip)
    latent = EmptyLatentImage(512, 512, 1)
    latent = KSampler(model, 12, 20, 8, 'euler', 'normal', conditioning, conditioning2, latent, 1)
    image = VAEDecode(latent, vae)
    SaveImage(image, 'ComfyUI')

def preview(task: Task, node_id: str, image: Image.Image):
    # 'KSampler.0'
    print(node_id)
wf.task.add_preview_callback(preview)

A stream form (sync/async iter) is also possible. But I'm not sure if it‘s worth the effort, as one can get similar results with a callback and a wait.

this code block doesn't make entire sense. Maybe I'm just a bit rusty with python. wf would only be valid within the context of the with block, right?

[Chaoses-Ib]

with block is justing calling wf.__enter__() and wf.__exit__(). And __exit__() for Workflow is just queuing the workflow to the server. So using it after __exit__() is fine.

But you are right, this is not a very intuitive API. Maybe I should move it as an arg of Workflow(), of allow set the callback in the with block.

[angelmankel]

Hi @Chaoses-Ib, I understand this is for runtime mode but I was just wondering if it was possible to get previews in real mode? Thanks in advance for any advice.

[Chaoses-Ib]

For now, you can directly set the hook in real mode:

from comfy_script.runtime.real import *
load(args=ComfyUIArgs('--preview-method', 'auto'))
from comfy_script.runtime.real.nodes import *

import comfy.utils

def hook(value, total, preview_image):
    print(value, total, preview_image)
comfy.utils.set_progress_bar_global_hook(hook)

with Workflow() as wf:
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    conditioning = CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,', clip)
    conditioning2 = CLIPTextEncode('text, watermark', clip)
    latent = EmptyLatentImage(512, 512, 1)
    latent = KSampler(model, 4, 20, 8, 'euler', 'normal', conditioning, conditioning2, latent, 1)
    image = VAEDecode(latent, vae)
    image = SaveImage(image, 'ComfyUI')
'''
1 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x27228632CE0>, 512)
2 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x27228632BC0>, 512)
3 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x27228633AC0>, 512)
...
18 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x272286489D0>, 512)
19 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x27228648D90>, 512)
20 20 ('JPEG', <PIL.Image.Image image mode=RGB size=64x64 at 0x27228728400>, 512)
'''

Note node id and prompt id are not available in real mode. If you need to identify nodes, you need some other methods, like:

import comfy.utils

node_id = None
def hook(value, total, preview_image):
    print(value, total, preview_image)
    print(node_id)
comfy.utils.set_progress_bar_global_hook(hook)

with Workflow() as wf:
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    conditioning = CLIPTextEncode('beautiful scenery nature glass bottle landscape, , purple galaxy bottle,', clip)
    conditioning2 = CLIPTextEncode('text, watermark', clip)
    latent = EmptyLatentImage(512, 512, 1)
    node_id = 'sampler'
    latent = KSampler(model, 3, 20, 8, 'euler', 'normal', conditioning, conditioning2, latent, 1)
    image = VAEDecode(latent, vae)
    image = SaveImage(image, 'ComfyUI')

[angelmankel]

Awesome, thanks for the quick response. I'll try this out.