llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard

Add backwards-compatibility for older model format

Open thement opened this issue 2 years ago • 1 comments

Add support for reading older model files so that people do not have to throw out ggml alpaca models.

thement avatar Mar 26 '23 14:03 thement

There has been a conversion script posted previously:

ggerganov/llama.cpp/issues/324#issuecomment-1476227818

with a link to this script:

https://gist.github.com/eiz/828bddec6162a023114ce19146cb2b82

I tried it once and it worked for me. Note that you might have to use the --n_parts 1 command-line arg when calling main with the converted params, as the script just generates a 1-part weights file.

marcom avatar Mar 26 '23 17:03 marcom

I think the conversion script should be included in the repository somewhere to be honest. It's worked for every old format model I've thrown at it and saved me a lot of time.

rabidcopy avatar Mar 26 '23 18:03 rabidcopy

I think the conversion script should be included in the repository somewhere to be honest. It's worked for every old format model I've thrown at it and saved me a lot of time.

yea, i suggested that in that thread, and got approved. no one actually bothered to do it. @eiz

Green-Sky avatar Mar 26 '23 18:03 Green-Sky

There has been a conversion script posted previously:

ggerganov/llama.cpp/issues/324#issuecomment-1476227818

with a link to this script:

https://gist.github.com/eiz/828bddec6162a023114ce19146cb2b82

I tried it once and it worked for me. Note that you might have to use the --n_parts 1 command-line arg when calling main with the converted params, as the script just generates a 1-part weights file.

OK, that was what I wanted to do in the first place but I could find the script. I'll make a merge request with this script instead.

thement avatar Mar 26 '23 19:03 thement