it’s so long a lime to wait while using a server

[niubi-AI]

[niubi-AI]

Any way to run as quickly as using llama.cpp directly? I need to save each input and response

[jessejohnson]

@dansinboy are you using the default server binary that comes with llama.cpp or a binding?

[niubi-AI]

@dansinboy are you using the default server binary that comes with llama.cpp or a binding?

you get the point , at first I used a binding mode with llama_cpp_python, worked badly. and then change into default server, wow, it works well right now ...

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.