StableCascade Type mismatch in gradio

In gradio_app/app.py, line 12

c_dtype = torch.bfloat16 if device.type == "cpu" else torch.float

Isn't the cuda device that uses bfloat16?

Feb 18 '24 01:02 ISDHN

Most likely, the same issue here: python gradio_app/app.py RUNNING ON: cuda Loading pipeline components...: 0%| | 0/6 [00:00<?, ?it/s]The config attributes {'c_in': 16} were passed to StableCascadeUnet, but are not expected and will be ignored. Please verify your config.json configuration file. Loading pipeline components...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:05<00:00, 1.00it/s] Keyword arguments {'ignore_mismatched_sizes': True} are not expected by StableCascadeDecoderPipeline and will be ignored. Loading pipeline components...: 0%| | 0/5 [00:00<?, ?it/s]The config attributes {'c_in': 4} were passed to StableCascadeUnet, but are not expected and will be ignored. Please verify your config.json configuration file. Loading pipeline components...: 40%|████████████████████████████████████████████████████████████████████████████████████████████████████████ | 2/5 [00:15<00:23, 7.82s/it] Traceback (most recent call last): File "D:\ai\StableCascade\gradio_app\app.py", line 14, in decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", torch_dtype=torch.half, low_cpu_mem_usage=False, ignore_mismatched_sizes=True) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 1263, in from_pretrained loaded_sub_model = load_sub_model( File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 531, in load_sub_model loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwargs) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\models\modeling_utils.py", line 743, in from_pretrained model, missing_keys, unexpected_keys, mismatched_keys, error_msgs = cls._load_pretrained_model( File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\models\modeling_utils.py", line 834, in _load_pretrained_model raise RuntimeError(f"Error(s) in loading state_dict for {model.class.name}:\n\t{error_msg}") RuntimeError: Error(s) in loading state_dict for StableCascadeUnet: size mismatch for embedding.1.weight: copying a param with shape torch.Size([320, 16, 1, 1]) from checkpoint, the shape in current model is torch.Size([320, 64, 1, 1]). You may consider adding ignore_mismatched_sizes=True in the model from_pretrained method.

Feb 19 '24 05:02 websepia

Ran into the following error when generating the image - occurs at image generation (I get the first 2 progress bars)

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 768.00 MiB. GPU 0 has a total capacty of 23.62 GiB of which 4.38 MiB is free. Including non-PyTorch memory, this process has 23.61 GiB memory in use. Of the allocated memory 22.72 GiB is allocated by PyTorch, and 423.33 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

tried big-big, small-big, big-small, small-small - all report the same memory usage.

changed line to 'c_dtype = torch.bfloat16 if device.type == "cpu" else torch.bfloat16'

Image generates now without issue (much quicker as well!!!)

Feb 20 '24 09:02 vkoski-godaddy

Most likely, the same issue here: python gradio_app/app.py RUNNING ON: cuda Loading pipeline components...: 0%| | 0/6 [00:00<?, ?it/s]The config attributes {'c_in': 16} were passed to StableCascadeUnet, but are not expected and will be ignored. Please verify your config.json configuration file. Loading pipeline components...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:05<00:00, 1.00it/s] Keyword arguments {'ignore_mismatched_sizes': True} are not expected by StableCascadeDecoderPipeline and will be ignored. Loading pipeline components...: 0%| | 0/5 [00:00<?, ?it/s]The config attributes {'c_in': 4} were passed to StableCascadeUnet, but are not expected and will be ignored. Please verify your config.json configuration file. Loading pipeline components...: 40%|████████████████████████████████████████████████████████████████████████████████████████████████████████ | 2/5 [00:15<00:23, 7.82s/it] Traceback (most recent call last): File "D:\ai\StableCascade\gradio_app\app.py", line 14, in decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", torch_dtype=torch.half, low_cpu_mem_usage=False, ignore_mismatched_sizes=True) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 1263, in from_pretrained loaded_sub_model = load_sub_model( File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 531, in load_sub_model loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwargs) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\models\modeling_utils.py", line 743, in from_pretrained model, missing_keys, unexpected_keys, mismatched_keys, error_msgs = cls._load_pretrained_model( File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\models\modeling_utils.py", line 834, in _load_pretrained_model raise RuntimeError(f"Error(s) in loading state_dict for {model.class.name}:\n\t{error_msg}") RuntimeError: Error(s) in loading state_dict for StableCascadeUnet: size mismatch for embedding.1.weight: copying a param with shape torch.Size([320, 16, 1, 1]) from checkpoint, the shape in current model is torch.Size([320, 64, 1, 1]). You may consider adding ignore_mismatched_sizes=True in the model from_pretrained method.

same issue

Feb 21 '24 02:02 MontaEllis

refer other issue fix this #58 update diffusers to another commit id pip install git+https://github.com/kashif/diffusers.git@a3dc21385b7386beb3dab3a9845962ede6765887 --force

have a try.

Feb 21 '24 07:02 xinzhuang

Most likely, the same issue here: python gradio_app/app.py RUNNING ON: cuda Loading pipeline components...: 0%| | 0/6 [00:00<?, ?it/s]The config attributes {'c_in': 16} were passed to StableCascadeUnet, but are not expected and will be ignored. Please verify your config.json configuration file. Loading pipeline components...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:05<00:00, 1.00it/s] Keyword arguments {'ignore_mismatched_sizes': True} are not expected by StableCascadeDecoderPipeline and will be ignored. Loading pipeline components...: 0%| | 0/5 [00:00<?, ?it/s]The config attributes {'c_in': 4} were passed to StableCascadeUnet, but are not expected and will be ignored. Please verify your config.json configuration file. Loading pipeline components...: 40%|████████████████████████████████████████████████████████████████████████████████████████████████████████ | 2/5 [00:15<00:23, 7.82s/it] Traceback (most recent call last): File "D:\ai\StableCascade\gradio_app\app.py", line 14, in decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", torch_dtype=torch.half, low_cpu_mem_usage=False, ignore_mismatched_sizes=True) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 1263, in from_pretrained loaded_sub_model = load_sub_model( File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 531, in load_sub_model loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwargs) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\models\modeling_utils.py", line 743, in from_pretrained model, missing_keys, unexpected_keys, mismatched_keys, error_msgs = cls._load_pretrained_model( File "D:\Users\spark\miniforge3\envs\StableCascade\lib\site-packages\diffusers\models\modeling_utils.py", line 834, in _load_pretrained_model raise RuntimeError(f"Error(s) in loading state_dict for {model.class.name}:\n\t{error_msg}") RuntimeError: Error(s) in loading state_dict for StableCascadeUnet: size mismatch for embedding.1.weight: copying a param with shape torch.Size([320, 16, 1, 1]) from checkpoint, the shape in current model is torch.Size([320, 64, 1, 1]). You may consider adding ignore_mismatched_sizes=True in the model from_pretrained method.

I don't think this issue is caused by the problem I mentioned.
The root of the shown traceback is line14:
decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", torch_dtype=torch.half and it has nothing to do with lilne12, which only infects line13

Feb 21 '24 08:02 ISDHN

refer other issue fix this #58 update diffusers to another commit id pip install git+https://github.com/kashif/diffusers.git@a3dc21385b7386beb3dab3a9845962ede6765887 --force

have a try.

Works for me. Thanks!

Feb 21 '24 11:02 websepia

refer other issue fix this #58 update diffusers to another commit id pip install git+https://github.com/kashif/diffusers.git@a3dc21385b7386beb3dab3a9845962ede6765887 --force

have a try.

This fix worked for me. Please update the README and huggingface with the correct version of diffusers.

Feb 26 '24 00:02 donbcolab

we fixed all errors with fp16 support : https://github.com/Stability-AI/StableCascade/issues/125

Mar 22 '24 03:03 FurkanGozukara

Type mismatch in gradio_app/app.py