ValueError: The current `device_map` had weights offloaded to the disk.

Open gitfans5 opened this issue 3 years ago • 1 comments

How to fix "ValueError: The current device_map had weights offloaded to the disk. Please provide an offload_folder for them. Alternatively, make sure you have safetensors installed if the model you are using offers the weights in this format."? when running "model = gal.load_model("standard")"? Thanks.

Dec 11 '22 13:12 gitfans5

It looks like the model cannot fit into the RAM + VRAM. What is the amount of free RAM & VRAM? You can reduce memory requirement by using float16 inference with load_model(..., dtype=torch.float16).

Dec 13 '22 15:12 mkardas