opt-175b model how to load model from disc.

[prof-schacht]

I tried to load the opt-175b model by using the following command:

python3 -m flexgen.flex_opt --model facebook/opt-175b --percent 0 0 100 0 100 0 --offload-dir ./tmp_offline

The issue I have is, that after converting the weights using Alpha to numpy as described in a folder. I'm not knowing how to define that the script should use the folder for loading the model.

I always get the error: OSError: facebook/opt-175b is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'

How do you have to define the command to load the locally stored weights of the opt-175b model?

[xangma]

The numpy weights seem to want to be put in ~/opt_weights/opt-175b-np?

[merrymercy]

Yes. The numpy weights should be put under ~/opt_weights/opt-175b-np/. You can check the downloaded weights of smaller models for the required format.