Zejun-Yang
单卡GPU 需要多大内存能跑起来
it took 16GB GPU memory, and 10GB shared memory to run audio2vid for 512x512 video. I tried to reduce the resolution to 256x256, only the shared memory usage is reduce to 5GB, which is not helpful. I can see the 'Pose2VideoPipeline' class took too much memory and training time. Any idea on reducing the memory usage? BTW, the default weight_dtype is fp16.
I can see vae, image_enc, reference_unet uses cuda in x class, and denoising_unet, pose_guider uses cpu in x class. Is that possible to change any of vae, image_enc, reference_unet to uses cpu to improve the memory usage without affecting the quality too much. And how to adjust to apply the above thoughts? Great Appreciate!
