MiniCPM-V icon indicating copy to clipboard operation
MiniCPM-V copied to clipboard

combine dtype and device to save CPU memory

Open whn09 opened this issue 1 year ago • 0 comments

Original code will load model to CPU memory in float32, and will require 36GB+ CPU memory. My modification will use less CPU memory (< 20GB).

whn09 avatar May 27 '24 05:05 whn09