kast424 comments

Repositories
Issues
Comments

Results 2 comments of


                                            kast424

AttributeError: 'NoneType' object has no attribute 'to'

I could see magnitude pruning work on the same mistral model.

gpu memory size recommended for pruning the llama2-7b-chat-hf model

Mine has 80GB of GPU RAM >>>>>NVIDIA A100 (and H100) GPU in Stanage has 80GB of GPU RAM still got this error. torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate...