huggingface
StableDiffusionXLPipeline.from_single_file Error
Describe the bug
version: 0.27.0, 0.27.1 when use StableDiffusionXLPipeline.from_single_file,error occur:
File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py", line 383, in encode_prompt
prompt_embeds = text_encoder(text_input_ids.to(device), output_hidden_states=True)
File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 822, in forward
return self.text_model(
File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 730, in forward
hidden_states = self.embeddings(input_ids=input_ids, position_ids=position_ids)
File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 229, in forward
position_embeddings = self.position_embedding(position_ids)
File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/modules/sparse.py", line 162, in forward
return F.embedding(
File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/functional.py", line 2210, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected tensor for argument #1 'indices' to have one of the following scalar types: Long, Int; but got torch.cuda.HalfTensor instead (while checking arguments for embedding)
but version 0.26.3 is OK
Reproduction
pipe = StableDiffusionXLPipeline.from_single_file('xxxxxxx.safetensors',
original_config_file='aaaa/sd_xl_base.yaml',
torch_dtype=torch.float16,
local_files_only=True,
use_safetensors=True,
add_watermarker=False,
# safety_checker=None,
# requires_safety_checker=False
)
print("22222222222222")
pipe.to('cuda')
prompt = "black denim,long jeans,1girl,datou,"
negative_prompt = '(worst quality:2.1),(low quality:2.1),(normal quality:2.1)'
# sdxl
image1 = pipe(
prompt=prompt,
negative_prompt=negative_prompt,
num_inference_steps=50,
eta=1.0,
).images[0]
image1.save("test.png")
Logs
No response
System Info
diffuser 0.27.0 0.27.1
Who can help?
No response
sorry I can't reproduce this, this runs for me
the config file is https://raw.githubusercontent.com/Stability-AI/generative-models/main/configs/inference/sd_xl_base.yaml
from diffusers import StableDiffusionXLPipeline
import torch
repo_id = "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0.safetensors"
pipe = StableDiffusionXLPipeline.from_single_file(repo_id,
original_config_file='sd_xl_base.yaml',
torch_dtype=torch.float16,
local_files_only=True,
use_safetensors=True,
add_watermarker=False,
)
pipe.enable_model_cpu_offload()
prompt = "black denim,long jeans,1girl,datou,"
negative_prompt = '(worst quality:2.1),(low quality:2.1),(normal quality:2.1)'
# sdxl
image1 = pipe(
prompt=prompt,
negative_prompt=negative_prompt,
num_inference_steps=50,
eta=1.0,
).images[0]
image1.save("yiyi_test_out.png")
sorry I can't reproduce this, this runs for me
the config file is https://raw.githubusercontent.com/Stability-AI/generative-models/main/configs/inference/sd_xl_base.yaml
from diffusers import StableDiffusionXLPipeline import torch repo_id = "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0.safetensors" pipe = StableDiffusionXLPipeline.from_single_file(repo_id, original_config_file='sd_xl_base.yaml', torch_dtype=torch.float16, local_files_only=True, use_safetensors=True, add_watermarker=False, ) pipe.enable_model_cpu_offload() prompt = "black denim,long jeans,1girl,datou," negative_prompt = '(worst quality:2.1),(low quality:2.1),(normal quality:2.1)' # sdxl image1 = pipe( prompt=prompt, negative_prompt=negative_prompt, num_inference_steps=50, eta=1.0, ).images[0] image1.save("yiyi_test_out.png")
what's your version? 0.26.3 is OK, but 0.27.0/0.27.1/0.27.2 occur error sd_xl_base_1.0.safetensors sd_xl_base.yaml
Unable to reproduce: https://colab.research.google.com/gist/sayakpaul/62e5e57f34161122b8a1439906b84c51/scratchpad.ipynb.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
closing now since we are not able to reproduce on our end if this is still an issue, please let us know and we will reopen it!
I am able to reproduce it with the following minimal code:
from diffusers import StableDiffusionXLPipeline import torch
pipeline = StableDiffusionXLPipeline.from_single_file( "/workspace/stable-diffusion-webui/models/Stable-diffusion/realismEngineSDXL_v30VAE.safetensors", torch_dtype=torch.float16 ).to("cuda")
image = pipeline(prompt="photo").images[0]
The venv is absl-py==1.4.0 accelerate==0.21.0 addict==2.4.0 aenum==3.1.15 aiofiles==23.2.1 aiohttp==3.8.5 aiosignal==1.3.1 altair==5.1.1 antlr4-python3-runtime==4.9.3 anyio==3.7.1 argon2-cffi==23.1.0 argon2-cffi-bindings==21.2.0 arrow==1.2.3 asttokens==2.2.1 async-lru==2.0.4 async-timeout==4.0.3 attrs==23.1.0 av==10.0.0 Babel==2.12.1 backcall==0.2.0 basicsr==1.4.2 beautifulsoup4==4.12.2 bleach==6.0.0 blendmodes==2022 boltons==23.0.0 cachetools==5.3.1 certifi==2023.7.22 cffi==1.15.1 chardet==5.2.0 charset-normalizer==3.2.0 clean-fid==0.1.35 click==8.1.7 clip==1.0 cmake==3.25.0 comm==0.1.4 contourpy==1.1.0 cssselect2==0.7.0 cycler==0.11.0 debugpy==1.6.7.post1 decorator==5.1.1 defusedxml==0.7.1 deprecation==2.1.0 diffusers==0.27.2 dynamicprompts==0.31.0 einops==0.4.1 exceptiongroup==1.1.3 executing==1.2.0 facexlib==0.3.0 fake-useragent==1.5.1 fastapi==0.94.0 fastjsonschema==2.18.0 ffmpy==0.3.1 filelock==3.9.0 filterpy==1.4.5 flatbuffers==23.5.26 fonttools==4.42.1 fqdn==1.5.1 frozenlist==1.4.0 fsspec==2023.6.0 ftfy==6.1.1 future==0.18.3 fvcore==0.1.5.post20221221 gdown==4.7.1 gfpgan==1.3.8 gitdb==4.0.10 GitPython==3.1.32 google-auth==2.22.0 google-auth-oauthlib==1.0.0 gradio==3.41.2 gradio_client==0.5.0 grpcio==1.57.0 h11==0.12.0 httpcore==0.15.0 httpx==0.24.1 huggingface-hub==0.23.0 idna==3.4 imageio==2.31.2 imageio-ffmpeg==0.4.8 importlib-metadata==6.8.0 importlib-resources==6.0.1 inflection==0.5.1 iopath==0.1.10 ipykernel==6.25.1 ipython==8.14.0 ipywidgets==8.1.0 isoduration==20.11.0 jedi==0.19.0 Jinja2==3.1.2 json5==0.9.14 jsonmerge==1.8.0 jsonpointer==2.4 jsonschema==4.19.0 jsonschema-specifications==2023.7.1 jupyter-events==0.7.0 jupyter-lsp==2.2.0 jupyter_client==8.3.1 jupyter_core==5.3.1 jupyter_server==2.7.3 jupyter_server_terminals==0.4.4 jupyterlab==4.0.5 jupyterlab-pygments==0.2.2 jupyterlab-widgets==3.0.8 jupyterlab_server==2.24.0 kiwisolver==1.4.5 kornia==0.6.7 lark==1.1.2 lazy_loader==0.3 lightning-utilities==0.9.0 lit==15.0.7 llvmlite==0.40.1 lmdb==1.4.1 lpips==0.1.4 lxml==4.9.3 lycoris-lora==2.2.0.post3 Markdown==3.4.4 markdown-it-py==3.0.0 MarkupSafe==2.1.3 matplotlib==3.7.2 matplotlib-inline==0.1.6 mdurl==0.1.2 mediapipe==0.10.3 mistune==3.0.1 mpmath==1.2.1 multidict==6.0.4 nbclient==0.8.0 nbconvert==7.8.0 nbformat==5.9.2 nest-asyncio==1.5.7 networkx==3.0 notebook_shim==0.2.3 numba==0.57.1 numexpr==2.8.5 numpy==1.23.5 oauthlib==3.2.2 omegaconf==2.2.3 open-clip-torch==2.20.0 opencv-contrib-python==4.8.0.76 opencv-python==4.8.0.76 orjson==3.9.5 overrides==7.4.0 packaging==23.1 pandas==2.1.0 pandocfilters==1.5.0 parso==0.8.3 pexpect==4.8.0 pickleshare==0.7.5 piexif==1.1.3 Pillow==9.5.0 PIMS==0.6.1 platformdirs==3.10.0 portalocker==2.7.0 prometheus-client==0.17.1 prompt-toolkit==3.0.39 protobuf==3.20.0 psutil==5.9.5 ptyprocess==0.7.0 pure-eval==0.2.2 py-cpuinfo==9.0.0 pyasn1==0.5.0 pyasn1-modules==0.3.0 pycparser==2.21 pydantic==1.10.12 pydub==0.25.1 Pygments==2.16.1 pyparsing==3.0.9 PySocks==1.7.1 python-dateutil==2.8.2 python-json-logger==2.0.7 python-multipart==0.0.6 pytorch-lightning==1.9.4 pytz==2023.3 PyWavelets==1.4.1 PyYAML==6.0.1 pyzmq==25.1.1 realesrgan==0.3.0 referencing==0.30.2 regex==2023.8.8 reportlab==4.0.4 requests==2.31.0 requests-oauthlib==1.3.1 resize-right==0.0.2 rfc3339-validator==0.1.4 rfc3986-validator==0.1.1 rich==13.5.2 rpds-py==0.10.0 rsa==4.9 safetensors==0.3.1 scikit-image==0.21.0 scipy==1.11.2 seaborn==0.13.2 semantic-version==2.10.0 Send2Trash==1.8.2 sentencepiece==0.1.99 six==1.16.0 slicerator==1.1.0 smmap==5.0.0 sniffio==1.3.0 sounddevice==0.4.6 soupsieve==2.4.1 stack-data==0.6.2 starlette==0.26.1 svglib==1.5.1 sympy==1.11.1 tabulate==0.9.0 tb-nightly==2.15.0a20230831 tensorboard-data-server==0.7.1 termcolor==2.3.0 terminado==0.17.1 thop==0.1.1.post2209072238 tifffile==2023.8.30 timm==0.9.2 tinycss2==1.2.1 tokenizers==0.13.3 tomesd==0.1.3 toml==0.10.2 tomli==2.0.1 toolz==0.12.0 torch==2.0.1+cu118 torchaudio==2.0.2+cu118 torchdiffeq==0.2.3 torchmetrics==1.1.1 torchsde==0.2.5 torchvision==0.15.2+cu118 tornado==6.3.3 tqdm==4.66.1 traitlets==5.9.0 trampoline==0.1.2 transformers==4.30.2 triton==2.0.0 typing_extensions==4.7.1 tzdata==2023.3 ultralytics==8.2.8 uri-template==1.3.0 urllib3==1.26.16 uvicorn==0.23.2 wcwidth==0.2.6 webcolors==1.13 webencodings==0.5.1 websocket-client==1.6.2 websockets==11.0.3 Werkzeug==2.3.7 widgetsnbextension==4.0.8 xformers==0.0.21 yacs==0.1.8 yapf==0.40.1 yarl==1.9.2 zipp==3.16.2 ZipUnicode==1.1.1
I am NOT getting this error with a fresh venv installed with the most recent versions available to pip. So you could close this bug report as fixed - but you might get this report again, because the venv posted above is in active use on many machines - it is the venv that stable diffusion A1111 webui creates.
