chatbot-ui-llama.cpp
chatbot-ui-llama.cpp copied to clipboard
Multi update of ChatBotUI (New prompt format, addressing issues, new features)
Hi,
I've made a few commits to improve this repository, namely:
1. Changed default language to English:
- Since most users prefer English, it makes sense to have it set as the default language. This change also addresses issue #1.
2. Added LLaMA-3.1 prompt format and ability to autofill the current date:
- LLaMA-3.1 models use a prompt format that is not included in the current version of ChatBotUI, so I added it.
- This format includes information about the current date within its system prompt. I added the special text
[@today_date], which, when included in any model's prompt format system prompt, will be replaced by the current date by ChatBotUI. - The original LLaMA-3.1 format only contains the current day's information; however, the current time and day of the week information may also be useful. Therefore,
[@today_date]includes all of this: the day of the week, followed by the date, followed by the current time.
3. Added Gemma prompt template:
- Added a prompt for another new popular model, Gemma. The system prompt is custom, as the original Gemma model does not use any system prompt. However, since the current date information might be helpful, a simple system prompt (similar to the LLaMA-3.1 one) is added.
4. Added generation speed indicator, changed footer:
- Added a generation speed indicator, which is included in the original LLaMA CPP server frontend UI but is missing in ChatBotUI. This change also addresses issue #2.
- Changed the footer to reflect that this project is a port of ChatBotUI for the LLaMA CPP server.
- Changed the plugin selection option from
ChatGPTtoLLaMA CPPsince this UI port uses LLaMA CPP, not ChatGPT.
5. Added fix for special failure scenario of LLaMA CPP:
- It seems that sometimes the LLaMA CPP completion endpoint either does not return the calculated generation speed information, or its response (prior to the parsing introduced by my code changes) isn't recognized as "done," so the information was not extracted. This was causing an error due to the
toFixedmethod call.