ddh0

@fakerybakery here is an example config.yml from Undi: https://huggingface.co/Undi95/Mixtral-8x7B-MoE-RP-Story/blob/main/config.yaml

I'm having a similar issue. I'm following the [guide](https://jan.ai/guides/quickstart/#1-create-a-model-folder) and I'm unable to get my model to show up in Jan, either on the models page or on the Hub....

Just pulled latest from master. When trying to convert from HF/safetensors to GGUF using `convert-hf-to-gguf.py` I get: ``` Loading model: Meta-Llama-3-8B-Instruct gguf: This GGUF file is for Little Endian only...

> template for llama 3 is > > --in-prefix " user " --in-suffix " assistant " -p "system You are a helpful, smart, kind, and efficient AI assistant. You always...

> > The instruct models need the `tokenizer.ggml.eos_token_id` to be 128009, or ``. > > @jxy Our comments were sent at the same time :) Yes, that's one of the...

@ggerganov Try `--vocab-type bpe` with convert-hf-to-gguf.py, that worked for me

I'd really love to see this merged, is there anything that needs to be done before that happens?

> This PR has now been open for 3 full months, without a single comment from any of the maintainers, despite it being the second-most upvoted PR in this repository,...

@abetlen @congson1293 > I checked the parameters in both __call__ and create_completion method but not see penalty_alpha param which represent for contrastive search decoding. Can you update the decoding strategy...

For reference I am using the chat template [described here](https://github.com/ggerganov/llama.cpp/issues/6747#issuecomment-2065013606), which seems to be working perfectly, other than the stopping issue. Here are the official JSON files under [meta-llama/Meta-Llama-3-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct): -...