Sankha Subhra Mullick

icon indicating an email [email protected]

icon location India icon location Ph.D. in Computer Science from Indian Statistical Institute. Former LinkedIn. currently at Dolby as Senior AI Researcher.

Results 2 comments of Sankha Subhra Mullick

Model size doubles after .merge_and_unload() and .save_pretrained()

I am having the same issue with Falcon 1b. The original model is about 2.3g on disk while the adapter is about 40m. After merging, the model is saved with...

Model size doubles after .merge_and_unload() and .save_pretrained()

Try with dtype=torch.bfloat16 (i.e. during model load for merging, assuming the original was already in half precision so is the lora), that solved the issue for me. I believe the...