ValeKnappich
ValeKnappich
Hi @kermitt2! Thanks for the reply. As the Issue #377 is from 2019, I assume the problem is not going to be resolved any time soon. Do you know any...
Thanks @kermitt2! If I decide to spend more time on this and find a decently working solution, I will post an update here.
@com3dian thanks for reaching out. Indeed, your approach works as long as the headings are numbered (I guess thats where the `
Any updates on this?
> Sorry for a newb question, I don't find an answer. I succeeded in launching the server with unquantised Mistral7B: > > ``` > python3 -m sglang.launch_server --model-path mistralai/Mistral-7B-Instruct-v0.2 --port...