sidtandon2014 comments

Repositories
Issues
Comments

Results 1 comments of


                                            sidtandon2014

Not able to load peft (promt-tuned) model in multi-gpu settings for inference

@geronimi73: Did you find any resolution for this? I think while loading weights to model (https://github.com/huggingface/peft/blob/v0.10.0/src/peft/peft_model.py#L730) is loading everything on GPU:0 because while loading the PEFT weights there are 8...