How to Load OpenELM Pre-training Checkpoints using Hugging Face AutoModelForCausalLM ?

[jasonkrone]

Hi there,

First, really admire the work on OpenELM! Thank you for making your models and code available.

Question regarding the pre-training checkpoints linked here: how can we convert these checkpoints into the format expected by AutoModelForCausalLM.from_pretrained?

I presume there's a script that was used for conversion of the final model weights into HF format, but I couldn't find it in the repo.

Would very much appreciate any help on this!

Best, Jason

[a154377713]

I have also encountered the same problem. Do you have a solution?

[jasonkrone]

I didn't wind up solving this but here's a reference that might be helpful https://github.com/foundation-model-stack/foundation-model-stack/blob/4349dacef63e86b6c1acdccb69b48fe562365bb2/fms/models/llama.py#L592

[athrvkk]

As a follow question - Are there any plans to push the model checkpoints to Huggingface hub for ease of access (like the Pythia suite of models)? It would really help the NLP community! Thanks in Advance!