flynn

brpc 已支持 IPv6，braft 有计划支持吗？

I want to deploy three models, one large language model occupying one GPU, one embedding model and one re-ranking model sharing one GPU, How can I do it？

1

there are two gpu device on the kubenetes node, the timeSlicing.replicas is two, the nvidia.com/gpu of large langurage model is two, the nvidia.com/gpu of other models are one, but the...

支持qwen1.5或者qwen2吗？

1

用 ppl.pmx Export 导出模型，有大量的警告， Warning: The shape interface of opmx::XX（如 ParallelEmbedding、ColumnParallelLinear、Reshape等） type is missing，用转出来的 onnx 格式的文件启动 ppl_llm_server，提示 unsupported op: domain[opmx], type[ParallelEmbedding]

feat: Increase API concurrency control to avoid service downtime

1

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...