VainF
Memory of Model Increases and Inference Stays the Same After Pruning
The memory of my pytorch model increases after I save it to my directory using torch.save(). Also, the inference of my model does not really speed up. Shouldn't it decrease the memory and increase inference since it is structured pruning?
Hi,
I have a similar issue. I used the high level pruner "MagnitudePruner" for Mask-rcnn pruning, with iterative_steps = 1. The number of model paramers is reducec from 44M to 15.5M.
However, the inference after pruning is getting slower.
Hi,
I have a similar issue. I used the high level pruner "MagnitudePruner" for Mask-rcnn pruning, with
iterative_steps = 1. The number of model paramers is reducec from 44M to 15.5M.However, the inference after pruning is getting slower.
Hi, Have you solved this problem? Now I have a similar problem. The inference time has not changed after pruning.
Hello, if your model cannot fully utilize GPUs (less than 100%), width pruning may not lead to a significant acceleration of your model. In this case, increasing the batch size can show some improvements in speed.
@ydneysay @Zhiwei-Zhai @kewang-seu Hi, have you solved the problem? In my case, the inference time increased after pruning.
