modelscope
RuntimeError: "exp_vml_cpu" not implemented for 'Half'
hi，dear 大佬，这个是啥问题啊，
>>> video = pipe(
... input_image=image.resize((512, 512)),
... num_frames=128, fps=30, height=512, width=512,
... motion_bucket_id=127,
... num_inference_steps=50,
... min_cfg_scale=2, max_cfg_scale=2, contrast_enhance_scale=1.2
... )
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/data/envs/diffsynth/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/data/DiffSynth-Studio/diffsynth/pipelines/stable_video_diffusion.py", line 158, in __call__
image_emb_clip_posi = self.encode_image_with_clip(input_image)
File "/data/DiffSynth-Studio/diffsynth/pipelines/stable_video_diffusion.py", line 53, in encode_image_with_clip
image = SVDCLIPImageProcessor().resize_with_antialiasing(image, (224, 224))
File "/dat/DiffSynth-Studio/diffsynth/pipelines/stable_video_diffusion.py", line 226, in resize_with_antialiasing
input = self._gaussian_blur2d(input, ks, sigmas)
File "/da/DiffSynth-Studio/diffsynth/pipelines/stable_video_diffusion.py", line 302, in _gaussian_blur2d
kernel_x = self._gaussian(kx, sigma[:, 1].view(bs, 1))
File "/datg/DiffSynth-Studio/diffsynth/pipelines/stable_video_diffusion.py", line 289, in _gaussian
gauss = torch.exp(-x.pow(2.0) / (2 * sigma.pow(2.0)))
Package Version
altair 5.3.0 attrs 23.2.0 blinker 1.8.2 Brotli 1.0.9 cachetools 5.3.3 certifi 2024.6.2 charset-normalizer 2.0.4 click 8.1.7 controlnet_aux 0.0.7 cupy-cuda12x 13.2.0 diffsynth 1.0.0 einops 0.8.0 fastrlock 0.8.2 filelock 3.13.1 fsspec 2024.6.0 gitdb 4.0.11 GitPython 3.1.43 gmpy2 2.1.2 huggingface-hub 0.23.4 idna 3.7 imageio 2.34.2 imageio-ffmpeg 0.5.1 importlib_metadata 8.0.0 Jinja2 3.1.4 jsonschema 4.22.0 jsonschema-specifications 2023.12.1 lazy_loader 0.4 markdown-it-py 3.0.0 MarkupSafe 2.1.3 mdurl 0.1.2 mkl-fft 1.3.8 mkl-random 1.2.4 mkl-service 2.4.0 mpmath 1.3.0 networkx 3.2.1 numpy 1.26.4 opencv-python 4.10.0.84 packaging 24.1 pandas 2.2.2 pillow 10.3.0 pip 24.0 protobuf 5.27.2 psutil 6.0.0 pyarrow 16.1.0 pydeck 0.9.1 Pygments 2.18.0 PySocks 1.7.1 python-dateutil 2.9.0.post0 pytz 2024.1 PyYAML 6.0.1 referencing 0.35.1 regex 2024.5.15 requests 2.32.2 rich 13.7.1 rpds-py 0.18.1 safetensors 0.4.3 scikit-image 0.24.0 scipy 1.13.1 sentencepiece 0.2.0 setuptools 69.5.1 six 1.16.0 smmap 5.0.1 streamlit 1.36.0 streamlit-drawable-canvas 0.9.3 sympy 1.12 tenacity 8.4.2 tifffile 2024.6.18 timm 1.0.7 tokenizers 0.19.1 toml 0.10.2 toolz 0.12.1 torch 2.0.1 torchaudio 2.0.2 torchvision 0.15.2 tornado 6.4.1 tqdm 4.66.4 transformers 4.41.2 triton 2.0.0 typing_extensions 4.11.0 tzdata 2024.1 urllib3 2.2.2 watchdog 4.0.1 wheel 0.43.0 zipp 3.19.2
上述代码源于md，https://modelscope.cn/models/ECNU-CILab/ExVideo-SVD-128f-v1
from diffsynth import save_video, ModelManager, SVDVideoPipeline
import torch, requests
from PIL import Image
# Load models
model_manager = ModelManager(torch_dtype=torch.float16, device="cuda",
model_id_list=["stable-video-diffusion-img2vid-xt", "ExVideo-SVD-128f-v1"])
pipe = SVDVideoPipeline.from_model_manager(model_manager)
# Generate a video
torch.manual_seed(0)
image = Image.open(requests.get("https://www.modelscope.cn/api/v1/studio/ECNU-CILab/ExVideo-SVD-128f-v1/repo?Revision=master&FilePath=images%2F0.png", stream=True).raw)
image.save("image.png")
video = pipe(
input_image=image.resize((512, 512)),
num_frames=128, fps=30, height=512, width=512,
motion_bucket_id=127,
num_inference_steps=50,
min_cfg_scale=2, max_cfg_scale=2, contrast_enhance_scale=1.2
)
save_video(video, "video.mp4", fps=30)
符合requirements中的 版本要求， torch>=2.0.0 cupy-cuda12x pip transformers controlnet-aux==0.0.7 streamlit streamlit-drawable-canvas imageio imageio[ffmpeg] safetensors einops sentencepiece
非常抱歉，我暂时无法根据您提供的信息定位问题，我们会在后续的工作中尝试复现，可能需要一段时间
The issue is in the _gaussian function.
Need to change this line:
# Convert sigma to float32 before performing the exponentiation
to
# Convert sigma to float32 before performing the exponentiation
sigma = sigma.to(torch.float32)
x = x.to(torch.float32)
