rdm->cuda out of memory

[fido20160817]

CUDA out of memory. Tried to allocate 568.00 MiB (GPU 0; 23.88 GiB total capacity; 16.92 GiB already allocated; 195.75 MiB free; 17.29 GiB reserved in total by PyTorch) when running "python scripts/knn2img.py --prompt "a happy bear reading a newspaper, oil on canvas""

[DongyangHuLi]

CUDA out of memory. Tried to allocate 568.00 MiB (GPU 0; 23.88 GiB total capacity; 16.92 GiB already allocated; 195.75 MiB free; 17.29 GiB reserved in total by PyTorch) when running "python scripts/knn2img.py --prompt "a happy bear reading a newspaper, oil on canvas""

Hi~ How many resolution pictures do you train and how much memory do you need?

[MaximClouser]

I was able to solve this issue by using half precision. Here is what I changed (all in knn2img.py):

1. I applied .half() to the model on line 313, and the clip_text_encoder on line 315.
2. I wrapped the sampling part of the code with with torch.cuda.amp.autocast(): on line 350.

[18222why]

2. with torch.cuda.amp.autocast():

THANKS