Xinyang Zhang

what happened if the server has internal error when using go-sse as client

3

here is my test code ` r, _ := http.NewRequestWithContext(c, http.MethodPost, "http://xxxx/v1/generate_stream", strings.NewReader(string(bodyS))) conn := sse.NewConnection(r) ubsubscribe := conn.SubscribeToAll(func(event sse.Event) { eventS, _ := json.Marshal(event) switch event.Type { case "cycles",...

feat: any plan to support NPU

1

### Feature request support Ascend NPU ### Motivation more and more developer using NPU, and vllm is also interesting in NPU ecosystem, https://github.com/vllm-project/vllm/issues/1606. ### Other _No response_

### Describe the bug I executed "TRUST_REMOTE_CODE=True openllm start /usr1/models/chatglm-6b" and model was loaded successfully, swagger was available, but got error using v1/chat/compelitioins, ``` Traceback (most recent call last): File...