MoonRide303

@taozhiyuai I currently use Q6_K from https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF and this Modelfile: ``` FROM ./Meta-Llama-3-8B-Instruct-Q6_K.gguf TEMPLATE """{{ if .System }}system {{ .System }}{{ end }}user {{ .Prompt }}assistant {{ .Response }}""" SYSTEM...

@chrisbward With corrected GGUF from QuantFactory it just stops after each answer, as it should: 

@chrisbward It means you're using GGUF with broken tokenizer. Try one from https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF repository (I use Q6_K), new versions with properly working tokenizer were uploaded there today.

@taozhiyuai It can do it, but not really reliably (2nd answer is not Polish):  SFAIK L3 wasn't trained much on other languages, I've seen information that's most of it...

Also `NotImplementedError: Architecture 'Phi3ForCausalLM' not supported!` from convert-hf-to-gguf.py.

I have to admit it seems to be pretty smart, even that smallest 3.8B - looks worthy of fully supporting it. Many 7B and bigger models struggled with those simple...

I am still learning basics of llama.cpp, so maybe I am missing something, but shouldn't there also be separate parameter for main to inject system message in proper place, while...

You might want to wait for https://github.com/ggerganov/llama.cpp/pull/6965 to be merged, too (should happen soon).

@JettHu it seems to be just a sampler in this PR, I don't see TCD scheduler included - am I missing something? Isn't TCD scheduler necessary too, for TCD method...

Workflow that works pretty fine for me (with SDXL-Lightning variant): [MoonRide SDXL-Lightning+TCD workflow v1.json](https://github.com/comfyanonymous/ComfyUI/files/14625183/MoonRide.SDXL-Lightning%2BTCD.workflow.v1.json)