Load Chat Template from `chat_template.jinja`

[dhbrojas]

Feature request

It seems recent transformers versions don't store the chat_template in tokenizer_config.json anymore but rather create a separate chat_template.jinja file.

TGI expects the template to be in tokenizer_config.json and doesn't try loading from chat_template.jinja. This causes issues in subsequent calls to /v1/chat/completions which error with {"error_type": "template_error", "error": "template not found"}.

Motivation

N/A

Your contribution

Can do if necessary

[bbkjunior]

I also experienced this. I had to manually edit tokenizer_config to make chat completions runnable on TGI

[fank]

Any workarounds for this?