Your question

How to use Controlnet for SDXL models, I've tried many wasy to use the workflow: SDXL checkpoint -> SDXL lora -> ControlNet But it failed, it shows the Error:

Error occurred when executing ControlNetApply: 'NoneType' object has no attribute 'copy' File "C:\Software\ComfyUI\ComfyUI-aki-v1.3\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "C:\Software\ComfyUI\ComfyUI-aki-v1.3\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "C:\Software\ComfyUI\ComfyUI-aki-v1.3\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "C:\Software\ComfyUI\ComfyUI-aki-v1.3\nodes.py", line 770, in apply_controlnet c_net = control_net.copy().set_cond_hint(control_hint, strength)

I've tried:

ControlNet loader(kohya_controllite_xl_canny) + Canny Preprocessor + ControlNet Apply, error
ControlNet loader(kohya_controllite_xl_canny) + Aux Prepocessor (Canny Preprocessor) + ControlNet Apply, error
Directly LLLIite loader, no error, but ControlNet not working

And all images will ensure is 1024*1024 before sending to the preprocessor. Do anyone knows how to fix it? Or other sample workflows that I can refer?

Thank you!

Logs

No response

Other

No response

Jul 15 '24 06:07 Pancat007

It seems we would need you to upload the entire workflow where the problem occurs in order to analyze the issue properly.

Jul 15 '24 14:07 ltdrdata

This issue is being marked stale because it has not had any activity for 30 days. Reply below within 7 days if your issue still isn't solved, and it will be left open. Otherwise, the issue will be closed automatically.

Sep 03 '24 11:09 github-actions[bot]

It's a thing. I've heard of several others also running into this.

ComfyUI Error Report

Error Details

Node Type: ControlNetApply
Exception Type: AttributeError
Exception Message: 'NoneType' object has no attribute 'copy'

Stack Trace

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 317, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 192, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\AIA\Tools\ComfyUI\ComfyUI\nodes.py", line 799, in apply_controlnet
    c_net = control_net.copy().set_cond_hint(control_hint, strength)
            ^^^^^^^^^^^^^^^^

Sep 07 '24 19:09 Derjyn

It's a thing. I've heard of several others also running into this.

ComfyUI Error Report

Error Details

Node Type: ControlNetApply
Exception Type: AttributeError
Exception Message: 'NoneType' object has no attribute 'copy'

Stack Trace

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 317, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 192, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\AIA\Tools\ComfyUI\ComfyUI\nodes.py", line 799, in apply_controlnet
    c_net = control_net.copy().set_cond_hint(control_hint, strength)
            ^^^^^^^^^^^^^^^^

What is your workflow?

Sep 08 '24 02:09 ltdrdata

It happens in any workflow, so that doesn't seem to be the culprit. What does seem to be the culprit, are the ControlNet models. After digging more, it would seem the developer of the various ControlNet models causing a ruccus is doing other things, I stopped digging after realizing the tiling models don't do what everyone thought they do anymore? I moved over to a working-as-expected ControlNet model and it seems to be fine.

Still, there could be a bug on the Comfy side. The same models don't trip an error in previous versions and that's hard to dismiss (and probably shouldn't be, if it's an actual ComfyUI issue).

Here's the log from a basic workflow that failed with the ControlNet model fed through Ultimate SD Upscale:

ComfyUI Error Report

Error Details

Node Type: ControlNetApply
Exception Type: AttributeError
Exception Message: 'NoneType' object has no attribute 'copy'

Stack Trace

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 317, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 192, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)

  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "D:\AIA\Tools\ComfyUI\ComfyUI\nodes.py", line 799, in apply_controlnet
    c_net = control_net.copy().set_cond_hint(control_hint, strength)
            ^^^^^^^^^^^^^^^^

System Information

ComfyUI Version: v0.2.2
Arguments: ComfyUI\main.py --windows-standalone-build --use-pytorch-cross-attention
OS: nt
Python Version: 3.11.9 (tags/v3.11.9:de54cf5, Apr 2 2024, 10:12:12) [MSC v.1938 64 bit (AMD64)]
Embedded Python: true
PyTorch Version: 2.3.1+cu121

Devices

Name: cuda:0 NVIDIA GeForce RTX 3070 : cudaMallocAsync
- Type: cuda
- VRAM Total: 8589279232
- VRAM Free: 3266113650
- Torch VRAM Total: 4160749568
- Torch VRAM Free: 57471090

Logs

2024-09-08 08:04:45,312 - root - INFO - Total VRAM 8191 MB, total RAM 32707 MB
2024-09-08 08:04:45,312 - root - INFO - pytorch version: 2.3.1+cu121
2024-09-08 08:04:45,312 - root - INFO - Set vram state to: NORMAL_VRAM
2024-09-08 08:04:45,312 - root - INFO - Device: cuda:0 NVIDIA GeForce RTX 3070 : cudaMallocAsync
2024-09-08 08:04:52,281 - root - INFO - Using pytorch cross attention
2024-09-08 08:05:04,734 - root - INFO - [Prompt Server] web root: D:\AIA\Tools\ComfyUI\ComfyUI\web
2024-09-08 08:05:04,734 - root - INFO - Adding extra search path checkpoints D:\AIA\Assets\Checkpoints\
2024-09-08 08:05:04,734 - root - INFO - Adding extra search path clip_vision D:\AIA\Assets\ClipVision\
2024-09-08 08:05:04,734 - root - INFO - Adding extra search path controlnet D:\AIA\Assets\ControlNet\
2024-09-08 08:05:04,734 - root - INFO - Adding extra search path embeddings D:\AIA\Assets\Embeddings\
2024-09-08 08:05:04,734 - root - INFO - Adding extra search path loras D:\AIA\Assets\LoRa\
2024-09-08 08:05:04,734 - root - INFO - Adding extra search path upscale_models D:\AIA\Assets\Upscale\
2024-09-08 08:05:04,734 - root - INFO - Adding extra search path vae D:\AIA\Assets\VAE\
2024-09-08 08:05:16,398 - root - INFO - 
Import times for custom nodes:
2024-09-08 08:05:16,398 - root - INFO -    0.0 seconds: D:\AIA\Tools\ComfyUI\ComfyUI\custom_nodes\sd-dynamic-thresholding
2024-09-08 08:05:16,398 - root - INFO -    0.0 seconds: D:\AIA\Tools\ComfyUI\ComfyUI\custom_nodes\websocket_image_save.py
2024-09-08 08:05:16,398 - root - INFO -    0.0 seconds: D:\AIA\Tools\ComfyUI\ComfyUI\custom_nodes\ComfyUI-mape-Helpers
2024-09-08 08:05:16,398 - root - INFO -    0.1 seconds: D:\AIA\Tools\ComfyUI\ComfyUI\custom_nodes\ComfyUI_UltimateSDUpscale
2024-09-08 08:05:16,398 - root - INFO -    0.2 seconds: D:\AIA\Tools\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts
2024-09-08 08:05:16,398 - root - INFO -    0.4 seconds: D:\AIA\Tools\ComfyUI\ComfyUI\custom_nodes\rgthree-comfy
2024-09-08 08:05:16,398 - root - INFO -    0.5 seconds: D:\AIA\Tools\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Manager
2024-09-08 08:05:16,398 - root - INFO -    0.8 seconds: D:\AIA\Tools\ComfyUI\ComfyUI\custom_nodes\comfyui_controlnet_aux
2024-09-08 08:05:16,398 - root - INFO -    3.9 seconds: D:\AIA\Tools\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Impact-Pack
2024-09-08 08:05:16,398 - root - INFO - 
2024-09-08 08:05:16,412 - root - INFO - Starting server

2024-09-08 08:05:16,412 - root - INFO - To see the GUI go to: http://127.0.0.1:8188
2024-09-08 08:13:10,217 - root - INFO - got prompt
2024-09-08 08:13:10,220 - root - ERROR - Failed to validate prompt for output 10:
2024-09-08 08:13:10,221 - root - ERROR - * ControlNetApply 2:
2024-09-08 08:13:10,221 - root - ERROR -   - Required input is missing: image
2024-09-08 08:13:10,221 - root - ERROR - Output will be ignored
2024-09-08 08:13:10,221 - root - WARNING - invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': '', 'extra_info': {}}
2024-09-08 08:15:48,854 - root - INFO - got prompt
2024-09-08 08:15:48,870 - root - ERROR - Failed to validate prompt for output 14:
2024-09-08 08:15:48,871 - root - ERROR - * KSampler 4:
2024-09-08 08:15:48,871 - root - ERROR -   - Required input is missing: positive
2024-09-08 08:15:48,872 - root - ERROR - Output will be ignored
2024-09-08 08:15:48,872 - root - ERROR - Failed to validate prompt for output 10:
2024-09-08 08:15:48,872 - root - ERROR - Output will be ignored
2024-09-08 08:15:48,872 - root - WARNING - invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': '', 'extra_info': {}}
2024-09-08 08:16:06,370 - root - INFO - got prompt
2024-09-08 08:16:11,570 - root - INFO - model weight dtype torch.float16, manual cast: None
2024-09-08 08:16:12,011 - root - INFO - model_type EPS
2024-09-08 08:17:08,296 - root - INFO - Using pytorch attention in VAE
2024-09-08 08:17:08,299 - root - INFO - Using pytorch attention in VAE
2024-09-08 08:17:26,674 - root - INFO - Requested to load SDXLClipModel
2024-09-08 08:17:26,674 - root - INFO - Loading 1 new model
2024-09-08 08:17:27,060 - root - INFO - loaded completely 0.0 1560.802734375 True
2024-09-08 08:17:29,338 - root - INFO - Using pytorch attention in VAE
2024-09-08 08:17:29,340 - root - INFO - Using pytorch attention in VAE
2024-09-08 08:17:32,993 - root - WARNING - Warning torch.load doesn't support weights_only on this pytorch version, loading unsafely.
2024-09-08 08:17:36,393 - root - INFO - Requested to load SDXL
2024-09-08 08:17:36,393 - root - INFO - Loading 1 new model
2024-09-08 08:17:38,027 - root - INFO - loaded completely 0.0 4897.0483474731445 True
2024-09-08 08:17:45,745 - root - INFO - Requested to load AutoencoderKL
2024-09-08 08:17:45,745 - root - INFO - Loading 1 new model
2024-09-08 08:17:46,674 - root - INFO - loaded completely 0.0 159.55708122253418 True
2024-09-08 08:17:49,679 - root - ERROR - error checkpoint does not contain controlnet or t2i adapter data D:\AIA\Assets\ControlNet\ICNXL\controlnetxlCNXL_bdsqlszTileReal.safetensors
2024-09-08 08:17:49,858 - root - ERROR - !!! Exception during processing !!! 'NoneType' object has no attribute 'copy'
2024-09-08 08:17:49,859 - root - ERROR - Traceback (most recent call last):
  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 317, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 192, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "D:\AIA\Tools\ComfyUI\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AIA\Tools\ComfyUI\ComfyUI\nodes.py", line 799, in apply_controlnet
    c_net = control_net.copy().set_cond_hint(control_hint, strength)
            ^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'copy'

2024-09-08 08:17:49,860 - root - INFO - Prompt executed in 103.48 seconds

Attached Workflow

Please make sure that workflow does not contain any sensitive information such as API keys or passwords.

{"last_node_id":15,"last_link_id":23,"nodes":[{"id":1,"type":"ControlNetLoader","pos":{"0":60,"1":940},"size":{"0":315,"1":58},"flags":{},"order":0,"mode":0,"inputs":[],"outputs":[{"name":"CONTROL_NET","type":"CONTROL_NET","links":[1],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"ControlNetLoader"},"widgets_values":["ICNXL\\controlnetxlCNXL_bdsqlszTileReal.safetensors"]},{"id":5,"type":"EmptyLatentImage","pos":{"0":1260,"1":480},"size":{"0":220,"1":106},"flags":{},"order":1,"mode":0,"inputs":[],"outputs":[{"name":"LATENT","type":"LATENT","links":[4],"shape":3}],"properties":{"Node name for S&R":"EmptyLatentImage"},"widgets_values":[768,768,1]},{"id":8,"type":"VAEDecode","pos":{"0":1260,"1":280},"size":{"0":220,"1":60},"flags":{},"order":8,"mode":0,"inputs":[{"name":"samples","type":"LATENT","link":9},{"name":"vae","type":"VAE","link":10}],"outputs":[{"name":"IMAGE","type":"IMAGE","links":[11,12,16],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"VAEDecode"}},{"id":3,"type":"CheckpointLoaderSimple","pos":{"0":60,"1":280},"size":{"0":315,"1":98},"flags":{},"order":2,"mode":0,"inputs":[],"outputs":[{"name":"MODEL","type":"MODEL","links":[2,17],"slot_index":0,"shape":3},{"name":"CLIP","type":"CLIP","links":[5,6],"slot_index":1,"shape":3},{"name":"VAE","type":"VAE","links":null,"shape":3}],"properties":{"Node name for S&R":"CheckpointLoaderSimple"},"widgets_values":["Pony\\cyberrealisticPony_v63.safetensors"]},{"id":2,"type":"ControlNetApply","pos":{"0":420,"1":920},"size":{"0":317.4000244140625,"1":98},"flags":{},"order":10,"mode":0,"inputs":[{"name":"conditioning","type":"CONDITIONING","link":7},{"name":"control_net","type":"CONTROL_NET","link":1},{"name":"image","type":"IMAGE","link":12}],"outputs":[{"name":"CONDITIONING","type":"CONDITIONING","links":[18],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"ControlNetApply"},"widgets_values":[1]},{"id":7,"type":"CLIPTextEncode","pos":{"0":60,"1":580},"size":{"0":320,"1":120},"flags":{},"order":6,"mode":0,"inputs":[{"name":"clip","type":"CLIP","link":6}],"outputs":[{"name":"CONDITIONING","type":"CONDITIONING","links":[8,19],"slot_index":0,"shape":3}],"title":"Negative Prompt","properties":{"Node name for S&R":"CLIPTextEncode"},"widgets_values":["(score_1, score_2, score_3, score_4, score_5),\n(source_furry, source_pony)"]},{"id":9,"type":"VAELoader","pos":{"0":1260,"1":380},"size":{"0":220,"1":60},"flags":{},"order":3,"mode":0,"inputs":[],"outputs":[{"name":"VAE","type":"VAE","links":[10,20],"shape":3,"slot_index":0}],"properties":{"Node name for S&R":"VAELoader"},"widgets_values":["sdxl.vae.safetensors"]},{"id":14,"type":"PreviewImage","pos":{"0":1240,"1":820},"size":{"0":210,"1":26},"flags":{},"order":12,"mode":0,"inputs":[{"name":"images","type":"IMAGE","link":15}],"outputs":[],"properties":{"Node name for S&R":"PreviewImage"}},{"id":15,"type":"UpscaleModelLoader","pos":{"0":420,"1":820},"size":{"0":315,"1":58},"flags":{},"order":4,"mode":0,"inputs":[],"outputs":[{"name":"UPSCALE_MODEL","type":"UPSCALE_MODEL","links":[21],"shape":3}],"properties":{"Node name for S&R":"UpscaleModelLoader"},"widgets_values":["DAT\\4xFFHQDAT.pth"]},{"id":13,"type":"UltimateSDUpscale","pos":{"0":900,"1":820},"size":{"0":315,"1":614},"flags":{},"order":11,"mode":0,"inputs":[{"name":"image","type":"IMAGE","link":16},{"name":"model","type":"MODEL","link":17},{"name":"positive","type":"CONDITIONING","link":18},{"name":"negative","type":"CONDITIONING","link":19},{"name":"vae","type":"VAE","link":20},{"name":"upscale_model","type":"UPSCALE_MODEL","link":21}],"outputs":[{"name":"IMAGE","type":"IMAGE","links":[15],"shape":3,"slot_index":0}],"properties":{"Node name for S&R":"UltimateSDUpscale"},"widgets_values":[1.5,834443546147574,"randomize",20,4,"euler","normal",0.2,"Linear",512,512,8,32,"None",1,64,16,32,true,true]},{"id":6,"type":"CLIPTextEncode","pos":{"0":60,"1":420},"size":{"0":320,"1":120},"flags":{},"order":5,"mode":0,"inputs":[{"name":"clip","type":"CLIP","link":5}],"outputs":[{"name":"CONDITIONING","type":"CONDITIONING","links":[7,23],"slot_index":0,"shape":3}],"title":"Positive Prompt","properties":{"Node name for S&R":"CLIPTextEncode"},"widgets_values":["(score_9, score_8_up, score_7_up, score_6_up),\n(photograph, cinematic, highly detailed),\n\na cute fantasy slime creature bouncing in a field,\nhellish landscape, medieval"]},{"id":4,"type":"KSampler","pos":{"0":920,"1":280},"size":{"0":320,"1":262},"flags":{},"order":7,"mode":0,"inputs":[{"name":"model","type":"MODEL","link":2},{"name":"positive","type":"CONDITIONING","link":23},{"name":"negative","type":"CONDITIONING","link":8},{"name":"latent_image","type":"LATENT","link":4}],"outputs":[{"name":"LATENT","type":"LATENT","links":[9],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"KSampler"},"widgets_values":[323040932712897,"randomize",20,4,"euler","normal",1]},{"id":10,"type":"PreviewImage","pos":{"0":1500,"1":280},"size":[340,340],"flags":{},"order":9,"mode":0,"inputs":[{"name":"images","type":"IMAGE","link":11}],"outputs":[],"properties":{"Node name for S&R":"PreviewImage"}}],"links":[[1,1,0,2,1,"CONTROL_NET"],[2,3,0,4,0,"MODEL"],[4,5,0,4,3,"LATENT"],[5,3,1,6,0,"CLIP"],[6,3,1,7,0,"CLIP"],[7,6,0,2,0,"CONDITIONING"],[8,7,0,4,2,"CONDITIONING"],[9,4,0,8,0,"LATENT"],[10,9,0,8,1,"VAE"],[11,8,0,10,0,"IMAGE"],[12,8,0,2,2,"IMAGE"],[15,13,0,14,0,"IMAGE"],[16,8,0,13,0,"IMAGE"],[17,3,0,13,1,"MODEL"],[18,2,0,13,2,"CONDITIONING"],[19,7,0,13,3,"CONDITIONING"],[20,9,0,13,4,"VAE"],[21,15,0,13,5,"UPSCALE_MODEL"],[23,6,0,4,1,"CONDITIONING"]],"groups":[],"config":{},"extra":{"ds":{"scale":1,"offset":[-313.09090909090907,-152.20661157024773]}},"version":0.4}

Sep 08 '24 15:09 Derjyn

I've tried running the 1000 commit and the 1500 commit, but I couldn't confirm that they worked properly for the model in question. Which commit are you referring to that you say worked normally before?

Sep 08 '24 16:09 ltdrdata

This issue is being marked stale because it has not had any activity for 30 days. Reply below within 7 days if your issue still isn't solved, and it will be left open. Otherwise, the issue will be closed automatically.

Oct 09 '24 11:10 github-actions[bot]

I think this is still an issue and should ideally be addressed, either through a fix or through documentation clarifying why it's happening and why it's an intended behavior.

It seems like this is only happening with certain Controlnets, and I am experiencing the issue trying to use SDXL IP adapters, such as ip-adapter-plus_sdxl_vit-h.safetensors.

It seems like maybe the intention is not to use the native controlnet nodes for ip-adapters, but that is not very clear to end users, especially people who are migrating from A1111/Forge, where you would normally load ip-adapters in the controlnet section.

Can anyone shine some light on whether this is intended behavior, or if it's a bug? Error report and workflow that will reproduce the bug shown below.

ComfyUI Error Report

Error Details

Node Type: ControlNetApplyAdvanced
Exception Type: AttributeError
Exception Message: 'NoneType' object has no attribute 'copy'

Stack Trace

  File "C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)

  File "C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\nodes.py", line 848, in apply_controlnet
    c_net = control_net.copy().set_cond_hint(control_hint, strength, (start_percent, end_percent), vae=vae, extra_concat=extra_concat)
            ^^^^^^^^^^^^^^^^

System Information

ComfyUI Version: v0.2.3-9-g0dbba9f
Arguments: ComfyUI\main.py --windows-standalone-build --listen
OS: nt
Python Version: 3.11.8 (tags/v3.11.8:db85d51, Feb 6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)]
Embedded Python: true
PyTorch Version: 2.4.1+cu121

Devices

Name: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
- Type: cuda
- VRAM Total: 25756696576
- VRAM Free: 17056576482
- Torch VRAM Total: 6979321856
- Torch VRAM Free: 31896546

Logs

2024-10-16 15:26:09,832 - root - INFO - Total VRAM 24564 MB, total RAM 98071 MB
2024-10-16 15:26:09,833 - root - INFO - pytorch version: 2.4.1+cu121
2024-10-16 15:26:09,833 - root - INFO - Set vram state to: NORMAL_VRAM
2024-10-16 15:26:09,834 - root - INFO - Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
2024-10-16 15:26:10,447 - root - INFO - Using pytorch cross attention
2024-10-16 15:26:12,650 - root - INFO - [Prompt Server] web root: C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\web
2024-10-16 15:26:12,654 - root - INFO - Adding extra search path checkpoints D:/Models/Image_Gen_Models\Models
2024-10-16 15:26:12,654 - root - INFO - Adding extra search path unet D:/Models/Image_Gen_Models\Models/unet
2024-10-16 15:26:12,654 - root - INFO - Adding extra search path configs D:/Models/Image_Gen_Models\Models
2024-10-16 15:26:12,654 - root - INFO - Adding extra search path vae D:/Models/Image_Gen_Models\Models/VAE
2024-10-16 15:26:12,654 - root - INFO - Adding extra search path loras D:/Models/Image_Gen_Models\LoRAs
2024-10-16 15:26:12,654 - root - INFO - Adding extra search path upscale_models D:/Models/Image_Gen_Models\Upscalers
2024-10-16 15:26:12,654 - root - INFO - Adding extra search path clip D:/Models/Image_Gen_Models\Clip
2024-10-16 15:26:12,654 - root - INFO - Adding extra search path embeddings D:/Models/Image_Gen_Models\Embeddings
2024-10-16 15:26:12,654 - root - INFO - Adding extra search path controlnet D:/Models/Image_Gen_Models\Controlnets
2024-10-16 15:26:12,655 - root - INFO - Adding extra search path ipadapter D:/Models/Image_Gen_Models\Controlnets
2024-10-16 15:26:13,461 - root - INFO - 
Import times for custom nodes:
2024-10-16 15:26:13,461 - root - INFO -    0.0 seconds: C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\custom_nodes\websocket_image_save.py
2024-10-16 15:26:13,461 - root - INFO -    0.0 seconds: C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\custom_nodes\mikey_nodes
2024-10-16 15:26:13,461 - root - INFO -    0.0 seconds: C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus
2024-10-16 15:26:13,462 - root - INFO -    0.0 seconds: C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_tinyterraNodes
2024-10-16 15:26:13,462 - root - INFO -    0.2 seconds: C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager
2024-10-16 15:26:13,462 - root - INFO - 
2024-10-16 15:26:13,464 - root - INFO - Starting server

2024-10-16 15:26:13,464 - root - INFO - To see the GUI go to: http://0.0.0.0:8188
2024-10-16 15:26:13,464 - root - INFO - To see the GUI go to: http://[::]:8188
2024-10-16 15:29:00,855 - root - INFO - got prompt
2024-10-16 15:29:00,945 - root - INFO - model weight dtype torch.float16, manual cast: None
2024-10-16 15:29:00,947 - root - INFO - model_type EPS
2024-10-16 15:29:01,864 - root - INFO - Using pytorch attention in VAE
2024-10-16 15:29:01,865 - root - INFO - Using pytorch attention in VAE
2024-10-16 15:29:02,919 - root - INFO - Requested to load SDXLClipModel
2024-10-16 15:29:02,920 - root - INFO - Loading 1 new model
2024-10-16 15:29:03,317 - root - INFO - loaded completely 0.0 1560.802734375 True
2024-10-16 15:29:03,509 - root - INFO - Requested to load SDXL
2024-10-16 15:29:03,509 - root - INFO - Loading 1 new model
2024-10-16 15:29:04,062 - root - INFO - loaded completely 0.0 4897.0483474731445 True
2024-10-16 15:29:09,488 - root - INFO - Requested to load AutoencoderKL
2024-10-16 15:29:09,488 - root - INFO - Loading 1 new model
2024-10-16 15:29:09,723 - root - INFO - loaded completely 0.0 159.55708122253418 True
2024-10-16 15:29:10,064 - root - INFO - Prompt executed in 9.21 seconds
2024-10-16 15:29:20,536 - root - INFO - got prompt
2024-10-16 15:29:20,541 - root - INFO - Prompt executed in 0.00 seconds
2024-10-16 15:29:30,558 - root - INFO - got prompt
2024-10-16 15:29:36,329 - root - INFO - Prompt executed in 5.77 seconds
2024-10-16 15:29:55,961 - root - INFO - got prompt
2024-10-16 15:29:55,994 - root - ERROR - error could not detect control model type.
2024-10-16 15:29:55,995 - root - ERROR - error checkpoint does not contain controlnet or t2i adapter data D:/Models/Image_Gen_Models\Controlnets\ip-adapter-plus_sdxl_vit-h.safetensors
2024-10-16 15:29:55,997 - root - ERROR - !!! Exception during processing !!! 'NoneType' object has no attribute 'copy'
2024-10-16 15:29:55,998 - root - ERROR - Traceback (most recent call last):
  File "C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xms99\ComfyUI_windows_portable\ComfyUI\nodes.py", line 848, in apply_controlnet
    c_net = control_net.copy().set_cond_hint(control_hint, strength, (start_percent, end_percent), vae=vae, extra_concat=extra_concat)
            ^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'copy'

2024-10-16 15:29:55,998 - root - INFO - Prompt executed in 0.03 seconds

Attached Workflow

Please make sure that workflow does not contain any sensitive information such as API keys or passwords.

{"last_node_id":32,"last_link_id":71,"nodes":[{"id":12,"type":"Reroute","pos":{"0":1880,"1":740},"size":[75,26],"flags":{},"order":15,"mode":0,"inputs":[{"name":"","type":"*","link":20}],"outputs":[{"name":"","type":"VAE","links":[18],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":16,"type":"Empty Latent Ratio Select SDXL","pos":{"0":440,"1":260},"size":{"0":269.66778564453125,"1":104.90866088867188},"flags":{},"order":0,"mode":0,"inputs":[],"outputs":[{"name":"LATENT","type":"LATENT","links":[25],"slot_index":0}],"title":"Aspect Ratio & Batch","properties":{"Node name for S&R":"Empty Latent Ratio Select SDXL"},"widgets_values":["1:1 [1024x1024 square]",1]},{"id":14,"type":"Reroute","pos":{"0":910,"1":40},"size":[75,26],"flags":{},"order":6,"mode":0,"inputs":[{"name":"","type":"*","link":66}],"outputs":[{"name":"","type":"MODEL","links":[29],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":8,"type":"VAEDecode","pos":{"0":2244,"1":98},"size":{"0":210,"1":46},"flags":{},"order":21,"mode":0,"inputs":[{"name":"samples","type":"LATENT","link":7},{"name":"vae","type":"VAE","link":18}],"outputs":[{"name":"IMAGE","type":"IMAGE","links":[9],"slot_index":0}],"properties":{"Node name for S&R":"VAEDecode"},"widgets_values":[]},{"id":17,"type":"Reroute","pos":{"0":1240,"1":620},"size":[75,26],"flags":{},"order":10,"mode":0,"inputs":[{"name":"","type":"*","link":26}],"outputs":[{"name":"","type":"LATENT","links":[44],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":23,"type":"Reroute","pos":{"0":1675.4324951171875,"1":616.268310546875},"size":[75,26],"flags":{},"order":18,"mode":0,"inputs":[{"name":"","type":"*","link":44}],"outputs":[{"name":"","type":"LATENT","links":[45],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":18,"type":"Reroute","pos":{"0":1580,"1":40},"size":[75,26],"flags":{},"order":12,"mode":0,"inputs":[{"name":"","type":"*","link":29}],"outputs":[{"name":"","type":"MODEL","links":[30],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":13,"type":"Reroute","pos":{"0":450,"1":740},"size":[75,26],"flags":{},"order":8,"mode":0,"inputs":[{"name":"","type":"*","link":19}],"outputs":[{"name":"","type":"VAE","links":[20,53],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":15,"type":"Reroute","pos":{"0":935,"1":622},"size":[75,26],"flags":{},"order":4,"mode":0,"inputs":[{"name":"","type":"*","link":25}],"outputs":[{"name":"","type":"LATENT","links":[26],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":21,"type":"Reroute","pos":{"0":896,"1":576},"size":[75,26],"flags":{},"order":9,"mode":0,"inputs":[{"name":"","type":"*","link":38}],"outputs":[{"name":"","type":"CONTROL_NET","links":[51],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":25,"type":"Reroute","pos":{"0":439,"1":800},"size":[75,26],"flags":{},"order":5,"mode":0,"inputs":[{"name":"","type":"*","link":47}],"outputs":[{"name":"","type":"IMAGE","links":[49],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":22,"type":"Reroute","pos":{"0":1233,"1":564},"size":[75,26],"flags":{},"order":17,"mode":0,"inputs":[{"name":"","type":"*","link":51}],"outputs":[{"name":"","type":"CONTROL_NET","links":[40],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":26,"type":"Reroute","pos":{"0":1230,"1":801},"size":[75,26],"flags":{},"order":11,"mode":0,"inputs":[{"name":"","type":"*","link":49}],"outputs":[{"name":"","type":"IMAGE","links":[54],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":27,"type":"Reroute","pos":{"0":1238,"1":701},"size":[75,26],"flags":{},"order":16,"mode":0,"inputs":[{"name":"","type":"*","link":53}],"outputs":[{"name":"","type":"VAE","links":[52],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":24,"type":"LoadImage","pos":{"0":17,"1":527},"size":{"0":315,"1":314},"flags":{},"order":1,"mode":0,"inputs":[],"outputs":[{"name":"IMAGE","type":"IMAGE","links":[47],"slot_index":0},{"name":"MASK","type":"MASK","links":null}],"properties":{"Node name for S&R":"LoadImage"},"widgets_values":["example.jpg","image"]},{"id":32,"type":"Reroute","pos":{"0":759,"1":127},"size":[75,26],"flags":{},"order":7,"mode":0,"inputs":[{"name":"","type":"*","link":69}],"outputs":[{"name":"","type":"CLIP","links":[70,71],"slot_index":0}],"properties":{"showOutputText":false,"horizontal":false}},{"id":4,"type":"CheckpointLoaderSimple","pos":{"0":10,"1":100},"size":{"0":315,"1":98},"flags":{},"order":2,"mode":0,"inputs":[],"outputs":[{"name":"MODEL","type":"MODEL","links":[66],"slot_index":0},{"name":"CLIP","type":"CLIP","links":[69],"slot_index":1},{"name":"VAE","type":"VAE","links":[19],"slot_index":2}],"properties":{"Node name for S&R":"CheckpointLoaderSimple"},"widgets_values":["SDXL\\juggernautXL_juggXIByRundiffusion.safetensors"]},{"id":19,"type":"ControlNetLoader","pos":{"0":425,"1":415},"size":{"0":315,"1":58},"flags":{},"order":3,"mode":0,"inputs":[],"outputs":[{"name":"CONTROL_NET","type":"CONTROL_NET","links":[38],"slot_index":0}],"properties":{"Node name for S&R":"ControlNetLoader"},"widgets_values":["ip-adapter-plus_sdxl_vit-h.safetensors"]},{"id":10,"type":"CLIPTextEncode","pos":{"0":912,"1":345},"size":{"0":400,"1":200},"flags":{},"order":14,"mode":0,"inputs":[{"name":"clip","type":"CLIP","link":71}],"outputs":[{"name":"CONDITIONING","type":"CONDITIONING","links":[41],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"CLIPTextEncode"},"widgets_values":["watermark, bad anatomy"]},{"id":6,"type":"CLIPTextEncode","pos":{"0":900,"1":120},"size":{"0":410,"1":170},"flags":{},"order":13,"mode":0,"inputs":[{"name":"clip","type":"CLIP","link":70}],"outputs":[{"name":"CONDITIONING","type":"CONDITIONING","links":[33],"slot_index":0}],"properties":{"Node name for S&R":"CLIPTextEncode"},"widgets_values":["An image of a person"]},{"id":9,"type":"SaveImage","pos":{"0":2504,"1":96},"size":{"0":210,"1":270},"flags":{},"order":22,"mode":0,"inputs":[{"name":"images","type":"IMAGE","link":9}],"outputs":[],"properties":{},"widgets_values":["ComfyUI"]},{"id":3,"type":"KSampler","pos":{"0":1828,"1":98},"size":{"0":315,"1":262},"flags":{},"order":20,"mode":0,"inputs":[{"name":"model","type":"MODEL","link":30},{"name":"positive","type":"CONDITIONING","link":64},{"name":"negative","type":"CONDITIONING","link":65,"slot_index":2},{"name":"latent_image","type":"LATENT","link":45}],"outputs":[{"name":"LATENT","type":"LATENT","links":[7],"slot_index":0}],"properties":{"Node name for S&R":"KSampler"},"widgets_values":[387182075546558,"randomize",40,4,"dpmpp_2m_sde_gpu","karras",1]},{"id":20,"type":"ControlNetApplyAdvanced","pos":{"0":1387,"1":118},"size":{"0":315,"1":186},"flags":{},"order":19,"mode":0,"inputs":[{"name":"positive","type":"CONDITIONING","link":33},{"name":"negative","type":"CONDITIONING","link":41},{"name":"control_net","type":"CONTROL_NET","link":40},{"name":"image","type":"IMAGE","link":54},{"name":"vae","type":"VAE","link":52,"shape":7}],"outputs":[{"name":"positive","type":"CONDITIONING","links":[64],"slot_index":0},{"name":"negative","type":"CONDITIONING","links":[65],"slot_index":1}],"properties":{"Node name for S&R":"ControlNetApplyAdvanced"},"widgets_values":[1,0,1]}],"links":[[7,3,0,8,0,"LATENT"],[9,8,0,9,0,"IMAGE"],[18,12,0,8,1,"VAE"],[19,4,2,13,0,"*"],[20,13,0,12,0,"*"],[25,16,0,15,0,"*"],[26,15,0,17,0,"*"],[29,14,0,18,0,"*"],[30,18,0,3,0,"MODEL"],[33,6,0,20,0,"CONDITIONING"],[38,19,0,21,0,"*"],[40,22,0,20,2,"CONTROL_NET"],[41,10,0,20,1,"CONDITIONING"],[44,17,0,23,0,"*"],[45,23,0,3,3,"LATENT"],[47,24,0,25,0,"*"],[49,25,0,26,0,"*"],[51,21,0,22,0,"*"],[52,27,0,20,4,"VAE"],[53,13,0,27,0,"*"],[54,26,0,20,3,"IMAGE"],[64,20,0,3,1,"CONDITIONING"],[65,20,1,3,2,"CONDITIONING"],[66,4,0,14,0,"*"],[69,4,1,32,0,"*"],[70,32,0,6,0,"CLIP"],[71,32,0,10,0,"CLIP"]],"groups":[],"config":{},"extra":{"ds":{"scale":0.611590904484147,"offset":[-74.63494176714066,236.5772535103933]}},"version":0.4}

Additional Context

Oct 16 '24 12:10 xms991

'NoneType' object has no attribute 'copy' using ControlNet for comfyUI

Your question

Logs

Other

ComfyUI Error Report

Error Details

Stack Trace

ComfyUI Error Report

Error Details

Stack Trace

ComfyUI Error Report

Error Details

Stack Trace

System Information

Devices

Logs

Attached Workflow

ComfyUI Error Report

Error Details

Stack Trace

System Information

Devices

Logs

Attached Workflow

Additional Context