ValeKnappich

Results 5 comments of ValeKnappich

Hi @kermitt2! Thanks for the reply. As the Issue #377 is from 2019, I assume the problem is not going to be resolved any time soon. Do you know any...

Thanks @kermitt2! If I decide to spend more time on this and find a decently working solution, I will post an update here.

@com3dian thanks for reaching out. Indeed, your approach works as long as the headings are numbered (I guess thats where the `

Any updates on this?

> Sorry for a newb question, I don't find an answer. I succeeded in launching the server with unquantised Mistral7B: > > ``` > python3 -m sglang.launch_server --model-path mistralai/Mistral-7B-Instruct-v0.2 --port...