goodglitch

Can this be fixed, please? I am using gguf format, so can't just change tokenizer_config.json I also have tried to modify text_generation.py but it doesn't fix the problem. Btw, in...

> For API I had to manually insert in completions.py the fields: 'skip_special_tokens': False, 'custom_stopping_strings': '""' > > as the other side doesnt insert those fields. I think the API...

> > > For API I had to manually insert in completions.py the fields: 'skip_special_tokens': False, 'custom_stopping_strings': '""' > > > as the other side doesnt insert those fields. I...

> > > > > For API I had to manually insert in completions.py the fields: 'skip_special_tokens': False, 'custom_stopping_strings': '""' > > > > > as the other side doesnt...

> did you try "mode": "chat-instruct", Thanks for the reply! The mode "chat-instruct" produced exactly the same results as "chat". However, just "instruct" has done the job)) Do you know...

For those who wonder you can use "/v1/internal/model/load" to load model. So I close the issue.

I am new to this library and thought that I just can't find this feature, but it looks like it is missing. So +1 for colored text input.

Thanks! I will try escape sequences. On Wed, Jul 31, 2024, 14:03 j0yk1ll ***@***.***> wrote: > I am new to this library and thought that I just can't find this...

You can't change token's size (and you don't want), but you can increase maximal number of tokens that LLM generates in response to your prompt: -n N, --n-predict N: Set...

Really waiting for the version bump to solve the problems with llama 3.1. Hope Oobabooga team would do it soon.