llama3.java icon indicating copy to clipboard operation
llama3.java copied to clipboard
Published 1 year ago • verified publisher icon mukel

Added llama.cpp-compatible http-server.

Open srogmann opened this issue 1 year ago • 1 comments

This PR contains a llama.cpp-compatible HTTP-server using the HTTP-server of the JDK. In my tests I used the HTML- and JS-resources of llama.cpp.

I wrote this implementation a few weeks ago so it may not be mergeable directly.

srogmann avatar Oct 05 '24 22:10 srogmann

I have added an OpenAI-compatible Chat Completions API (with streaming) along with a HTTP session. However, the code is now more extensive (not using DTOs/JAX-RS, but also without dependencies).

srogmann avatar Nov 23 '24 21:11 srogmann