cOng
Hi @christobill, could you help to test with your models?
> I think llama2 also have the same situation. So for llama2` ` is the stop word? Test on LLAMA2, The prompt: `[INST] hi. Evaluate translation from English #This section...
HI, I mean the official stop tokens is indeed ``, but as you can see the conversation class has defined llama2 template with ` ` as stop string, for compatibility,...
Hi @infwinston , this PR is ready to be merged, could you help to have a final reviews and merged, so as the associated doc PR https://github.com/lm-sys/FastChat/pull/3139 These features should...
> @congchan Could you add the deepspeed zero3 support on the triain_with_template? > > Do you think if it should add > > ``` > if trainer.is_deepspeed_enabled: > trainer.save_model() >...
It is actually 1024. Can the team share guides to fine-tune 8K context length?
Hi, may I suggest to support **copy** the whole conversation history as well?
Run into the same issue for normal decoder only model
Hi guys, I encountered the same error for version 4.41.2. I am confused which package dose `flash_attn` refered to? I have tried install both of the two packages (xformers and...
Hi, I encounter similar issues, I am not sure if this is related with HF, since the value error arise for vllm>=0.7.3