cortex.cpp icon indicating copy to clipboard operation
cortex.cpp copied to clipboard

bug: slow response after terminate the conversation

Open cht132 opened this issue 1 year ago • 0 comments

Describe the bug close / terminate the conversation of the streaming output will introduce slow performance of new chat

To Reproduce Steps to reproduce the behavior:

  1. start the check enable with stream
  2. when nitro streaming text output, close the chat from client
  3. in server log, it displayed "Task completed, release it - llamaCPP.cc:416"
  4. start the new chat, it will be slow response until restart nitro from server

Desktop (please complete the following information):

  • OS: Mac
  • Browser chrome

cht132 avatar Apr 10 '24 02:04 cht132