Cleanup (for VRAM) if idle

[liusida]

This PR introduces a new function, cleanup_if_idle, which is triggered in two scenarios: when a user leaves and when the queue is updated. Its primary purpose is to efficiently release unused resources, ensuring better availability for other applications running on the same machine.

Related Issue: Fixes #3192

[comfyanonymous]

Your way wouldn't be good for people who use ComfyUI as a backend.

I think it would be better to add a button in the UI to do that, there's already a route for this https://github.com/comfyanonymous/ComfyUI/blob/master/server.py#L506

Send a post to http://127.0.0.1:8188/free with: {"unload_models": true, "free_memory": false}

unload_models: unload models from ram to vram free_memory: unload all workflows and models from both vram and ram (doesn't work that well because of pytorch caching)

[liusida]

Your way wouldn't be good for people who use ComfyUI as a backend.

Understand. Didn't notice that scenario...

But I'm pretty sure that people wouldn't like to click a button before leaving every time.

Wait, you mean people only use ComfyUI's REST API and develop their own clients, right? What about keeping a flag (a member variable in 'server') for that, and add another endpoint like '/serve' to activate/deactivate that mode. So we don't clean up if serve_mode==True.

If you think adding a flag is a good idea, I will implement that and commit again.

I think it would be better to add a button in the UI to do that, there's already a route for this https://github.com/comfyanonymous/ComfyUI/blob/master/server.py#L506

Send a post to http://127.0.0.1:8188/free with: {"unload_models": true, "free_memory": false}

unload_models: unload models from ram to vram free_memory: unload all workflows and models from both vram and ram (doesn't work that well because of pytorch caching)

Thanks for the input.

[liusida]

Seems this is not the preferred method. I'll close the PR for now. Thanks.